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Executive  Summary 


Background 

Over^  the  past^  decade,  military  personnel  research 
organizations  have  designed  and  developed  a  number  of  computerized 
tests  or  test  batteries  to  improve  selection  and  classification 
decisions  for  enlisted  personnel.  Two  of  the  most  important  of 
these  batteries  are  the  Computerized  Adaptive  Testing  version  of 
the  Armed  Services  Vocational  Aptitude  Battery  (CAT-ASVAB)  and  the 
Enhanced  Computer-Administered  Tests  battery  (ECAT) . 

Several  studies  have  used  these  batteries  to  investigate  gains 
in  predictive  validity  for  predicting  school  or  job  performance. 
However,  these  studies  focus  on  differences  in  predictive  validity 
and  implicitly  assume  the  use  of  a  selection  model  which  evaluates 
all  applicants  for  a  single  job.  They  do  not  measure  gains  in 
efficiency  due  to  considering  personnel  for  a  variety  of 
assignments,  i.e.,  personnel  classification. 

The^  present  study  compares  the  classification  efficiency  of 
eight  unique  alternative  combinations  of  individual  tests  from  the 
ASVAB,  ECAT,  and  CAT-ASVAB.  These  candidate  batteries  differ  in 
terms  of  abilities  measured,  mode  of  administration,  and  test  time. 
In  addition  to  examining  classification  efficiency,  this  study  also 
evaluates  the  utility  of  each  battery. 

Hypotheses 


The  primary  hypothesis  of  this  study  predicts  that  changes  in 
abilities  measured,  mode  of  administration,  and  test  time  can 
increase  the  classification  efficiency  of  eight  specified  test 
batteries.  In  addition,  we  tested  six  hypotheses  that  address 
comparisons  between  specified  pairs  of  these  batteries.  The 
comparisons  examine  the  effects  of  mode  of  administration  and/or 
abilities  measured  on  classification  efficiency.  For  these  six 
we  hypothesized  that  both  computerization  and  increases  in 
battery  complexity  would  increase  efficiency.  Since  classification 
utility  is  a  function  of  classification  efficiency,  improvements  in 
®^ficiency  should,  in  turn,  lead  to  corresponding  improvements  in 


Approach 


The  following  seven  steps  provide  an  overview  of  our  approach 
to  evaluate  these  candidate  batteries:  l)  First,  we  used  analytic 
techniques  to  create  validity  and  intercorrelation  matrices  for 
each  candidate  test  battery.  2)  From  the  matrices  of  each  battery, 
we  developed  regression  equations  to  predict  performance  in  each 
school.  3)  Next,  we  used  these  regression  equations  to  compute 
p®rformance  scores  for  members  of  several  randomly 
selected  applicant  samples.  4)  For  each  sample,  we  used  these 
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scores  to  optimally  assign  a  specified  portion  of  applicants  to 
schools.  5)  We  summed  the  individual  predicted  performance  scores 
associated  with  the  optimal  assignment  of  each  sample  to  compute  a 
mean  predicted  performance  score  for  each  battery.  6)  Next,  we 
averaged  the  mean  predicted  performance  scores  (across  samples)  to 
obtain  overall  means  for  each  test  battery.  7)  We  used  these 
results,  based  on  18  schools,  to  estimate  the  expected  utility  of 
each  candidate  battery  for  all  military  occupational  specialties 
(MOSs) . 

Findings 


As  expected,  the  eight  candidate  batteries  differed 
significantly  in  terms  of  classification  efficiency.  Differences 
among  the  batteries  accounted  for  about  50%  of  the  total  variation 
in  efficiency.  This  translates  into  a  23%  improvement  in  predicted 
performance  from  the  least  to  the  most  effective  of  the  eight 
batteries.  In  terms  of  relative  contribution,  it  appears  that  the 
greatest  improvement  results  from  increasing  the  abilities 
measured,  followed  by  mode  of  administration,  with  test  time 
contributing  the  least. 

Evaluation  of  the  paired  comparisons  also  yielded 
statistically  significant  findings,  supporting  all  six  hypotheses. 
These  six  comparisons  demonstrate  the  combined  benefits  of  a 
computerized  mode  of  administration  and  inclusion  of  additional 
measures . 

since  all  six  comparisons  produced  significant  differences,  we 
computed  the  dollar-based  utilities  for  all  comparisons.  The  four 
hypotheses  comparing  paper ~and~pencil  with  computerized  batteries 
produced  utilities  (in  net  present  value  terms)  that  ranged  from 
$6.8  to  $11.6  billion.  These  were  the  four  largest  utilities  in 
the  study.  Among  the  other  two  comparisons,  one  compared  two 
paper-and-pencil  tests  and  the  other  compared  two  computerized 
tests.  The  utilities  for  these  amounted  to  $2.3  billion  and  $3.2 
billion,  respectively. 

Conclusions 

The  results  of  the  present  study  support  the  following 
conclusions: 

1)  Use  of  a  computerized  mode  of  administration  and/or  computer- 
adaptive  tests  increase  classification  efficiency  and  the 

concomitant  classification  utility  over  that  of  a  paper  and 
pencil  mode  of  administration. 

2)  Inclusion  of  additional  tests  increase  classification 
efficiency  and,  in  turn,  increase  classification  utility,  over 
that  of  the  basic  ASVAB  or  CAT-ASVAB. 
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3)  Although  we  did  not  specifically  test  the  significance  of 
manipulating  test  time  among  the  candidate  batteries,  test 

increase  classification  efficiency  and 
utility.  However,  this  increase  is  small,  relative  to 
increases  due  to  test  computerization  and  expansion. 

<“  “T-ASVAB)  alone  to  an  ASVAB 
(or  CAT-ASVAB)  plus  full  ECAT  battery,  holding  test  time 

comparisons  we  do  make,  it  appears 
that  the  addition  of  ECAT  to  an  ASVAB  (or  CAT-ASVAB)  battery 
provides  a  substantial  gain  in  classification  efficiency. 

assignment  conditions  used  in  this  study,  using  the 

instead  of  using  p&p  ASVAB  and  three 
ECAT  tests  provided  the  largest  gain  in  utility.  in  this 

at”*?h^n?°Si' ®®^^™ated  the  dollar  value  of  the  utility  gain 
at  about  $11.6  billion  or  about  4.1%  of  the  DoD  expenditures 

^hL  ^  ®2).  Since  DOD  would  ^cumulate 

over  several  years,  the  present  value  of  the 
.  cash  flow  would  amount  to  about  $504 

amounts  to  0.178%  of  DoD  expenditures 

Recommendations 

£utur??Jsearch  sholiSl“®®  recommended  that 

construct  test  batteries  and  regression 
.  spultaneously  maximize  classification 
a^^^oiency  while  minimizing  adverse  impact. 

fvl=??n«  incremental  classification  efficiency  of  the 

i^  ^rd^?  existing  ASVAB  (or  CAT-ASVAB) 

^  Identify  improvements  in  personnel  classification 

Se^etopmen^"”^  additional  time  or  funding  on  test 


non-dollar  job  values  to  more 
adequately  reflect  the  value  DoD  places  on  military  jobs. 

SSacr  tK?®^e  analyses  to  investigate  the  cumulative 

J«u!ts  o/utfuty“’an.?ysn!™‘"''*  °n  the 
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Introduction 


Background 

Over  the  past  decade,  military  personnel  research 
organizations  have  designed  and  developed  a  number  of  computerized 
tests  or  test  batteries  to  improve  selection  and  classification 
decisions  for  enlisted  personnel.  The  most  prominent  of  these  is 
CAT-ASVAB,  a  computerized,  adaptive  version  of  the  Armed  Services 
Vocational  Aptitude  Battery. 

In  addition  to  CAT-ASVAB,  another  important  battery  of 
computerized  tests  is  the  Enhanced  Computer  Administered  Tests 
(ECAT)  battery.  The  tests  in  this  joint-service  battery  measure 
additional  abilities  which  could  augment  tests  included  in  ASVAB  or 
CAT-ASVAB.  (See  Wolfe,  Alderton,  Larson,  and  Held,  in  press,  for  a 
description  of  the  development  of  the  ECAT  battery . ) 

Several  studies  have  investigated  the  gain  in  predictive 
validity  resulting  from  using  ECAT  in  addition  to  ASVAB  for 
predicting  school  or  job  performance  (e.g.,  Wolfe  et  al.,  in 
press) .  Abrahams,  Pass,  Kusulas,  Cole,  and  Kieckhaefer  (1993) 
summarize  many  of  these  studies  and  present  their  own  results  oii 
the  incremental  validity  of  ECAT.  Essentially,  all  of  these 
studies  show  only  small  increments  in  predictive  validity  when 
averaging  ECAT  gains  over  a  set  of  schools  or  jobs.  The  studies 
focus  on  differences  in  predictive  validity,  and  implicitly  assume 
the  use'  of  a  selection  model  which  evaluates  all  applicants  for  a 
single  job.  They  do  not  measure  gains  in  efficiency  due  to 
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considering  personnel  for  a  variety  of  assignments,  that  is, 
personnel  classification. 

Several  theoretical  papers  demonstrate  the  importance  of 
evaluating  the  classification  efficiency  of  test  batteries 
(Brogden,  1951,  1959;  Horst,  1954) .  Brogden's  (1959) 
classification  model  indicates  that  classification  efficiency  is  a 
function  of  the  following  factors:  (1)  the  validity  of  the 
predictor  equations,  (2)  the  intercorrelation  of  predicted 
criterion  scores,  (3)  the  selection  ratio,  or  percent  rejected,  and 
(4)  the  number  of  possible  assignment  categories.  Since  the 
military  does,  in  fact,  evaluate  candidates  for  a  wide  variety  of 
occupational  specialties,  the  military  should  also  assess  the 
benefits  to  classification  efficiency  that  alternative  tests  could 
provide . 

A  number  of  studies  have  focused  on  the  classification 
efficiency  of  the  military's  classification  system.  Johnson  and 
Zeidner  (1990)  completed  a  series  of  reports  and  studies  on  the 
classification  efficiency  and  utility  of  the  ASVAB  as  well  as  the 
ASVAB  augmented  with  new  tests  (including  some  ECAT  tests) 
developed  for  Project  A.  They  document  the  potential  for 
substantially  improving  the  classification  efficiency  and  utility 
of  the  military  personnel  assignment  (or  utilization)  system 
(Zeidner  &  Johnson,  1989a,  1989b;  Johnson  &  Zeidner,  1990;  Johnson, 
Zeidner,  &  Scholarios,  1990).  Peterson,  Oppler,  and  Rosse  (1992) 
also  investigated  the  differential  validity  of  the  ASVAB  and  ECAT 
tests. 
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Purpose 

To  date,  no  studies  compare  the  classification  utility  of  the 
unique  alternative  combinations  of  tests  from  the  ASVAB,  ECAT,  and 
CAT-ASVAB  described  in  the  contract  statement  of  work  (SOW) .  These 
"candidate”  or  alternative  batteries  differ  in  various  ways 
includinq  abilities  measured,  and  method  of  administration*  The 
method  of  administration  includes  paper  &  pencil  (P&P) ,  computer- 
administered  (with  and  without  a  response  pedestal) ,  and  computer- 
adaptive.  The  types  of  abilities  include  verbal,  math,  spatial, 
perceptual,  memory,  and  psychomotor.  These  differences,  in  the 
method  of  administration  and  abilities  measured,  permit  systematic 
investiqation  of  a  number  of  hypotheses  concerninq  the 
classification  utility  of  alternative  test  combinations.  The  SOW 
presents  these  hypotheses  as  paired  comparisons  of  alternative 
combinations  (i.e.,  batteries)  of  tests  which  vary  in  terms  of 
abilities  measured  and  method  of  administration.  In  addition  to 
these  comparisons,  the  SOW  also  requires  that  we  simultaneously 
determine  the  optimal  allocation  of  total  test  battery  time  to 
individual  tests  to  maximize  classification  efficiency. 

Oveirviev  of  the  Study 

To  evaluate  the  classification  utility  of  each  alternate 
combination  of  tests,  we  performed  three  basic  steps.  In  the  first 
step,  we  generated  optimal  regression  equations.  The  second  step 
employed  predicted  performance,  estimated  from  these  equations  to 
optimally  assign  individuals  to  technical  schools.  Finally,  in  the 
third  step,  we  applied  a  utility  index  (SDy)  to  the  assignment 
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solutions  and  compared  the  classification  utility  of  alternate  test 
batteries . 

In  the  military,  assignment  to  a  technical  school  actually 
indicates  assignment  to  an  occupational  specialty  or  job. 
Typically,  there  is  a  unique  entry-level  specialty  associated  with 
each  school.  In  this  study,  we  use  the  terms  job,  school,  and 
occupational  specialty  interchangeably. 

As  in  other  research  studies  (e.g.,  Schmidt,  Hunter,  &  Dunn, 
1987) ,  the  present  study  employed  methods  which,  given  a  fixed  set 
of  assignment  quotas,  assign  individuals  to  maximize  predicted 
performance.  That  is,  we  assigned  each  person  to  one  job  such  that 
the  sum  of  the  predicted  performance  scores  over  all  people  in  the 
assigned  jobs  is  maximized.  In  practice,  however,  military 
assignment  policies  encompass  a  number  of  other  factors  (e.g., 
travel  costs  and  EEO  objectives)  that  constrain  the  possible 
assignments  and  the  concomitant  classification  utility.  For  this 
reason,  policy  makers  might  consider  the  resulting  utilities  in 
this  study  as  maximum  possible  values.  Nevertheless,  we  assume 
that  the  use  of  maximum  values  would  not  affect  the  relative 
f ®i’6nces  between  the  classification  utilities  observed  for 
alternative  batteries.  In  conducting  the  utility  analyses  in  this 
study,  we  assumed  that  the  proportional  difference  in  the  dollar 
benefit  between  alternate  batteries  would  not  change  if  put  into 
operational  use . 

It  is  important  to  note  that  the  results  of  this  study  alone 
are  not  intended  to  serve  as  the  basis  for  specific  changes  to  the 
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military's  operational  selection  and  classification  system.  This 
study's  results  provide  an  estimate  of  the  potential  relative  gain 
achieved  in  classification  utility  by  using  test  batteries  other 
than  the  ASVAB.  Such  information  is  useful  for  determining  the 
future  direction  of  efforts  to  improve  the  military's  selection  and 
classification  system.  In  summary,  this  report  documents  our 
evaluation  of  the  relative  classification  utility  of  various  test 
combinations  that  differ  from  the  current  ASVAB  in  terms  of  testing 
time,  method  of  administration,  and  abilities  measured. 
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Method 


Before  addressing  the  specific  procedures  involved  in  this 
study,  the  following  seven  steps  summarize  these  procedures:  1) 
First,  we  used  analytic  techniques  to  create  validity  and 
intercorrelation  matrices  for  each  candidate  test  battery.  2)  From 
the  matrices  of  each  battery,  we  developed  regression  equations  to 
predict  performance  in  each  school.  3)  Next,  we  used  these 
regression  equations  to  compute  predicted  performance  scores  for 
members  of  several  randomly  selected  applicant  samples.  4)  For 
each  sample,  we  used  these  scores  to  optimally  assign  a  specified 
portion  of  applicants  to  schools.  5)  We  used  the  individual 
predicted  performance  scores,  associated  with  the  optimal 
assignment  of  each  sample,  to  compute  a  mean  predicted  performance 
score  for  each  battery.  6)  Next,  we  averaged  the  mean  predicted 
performance  scores  (across  samples)  to  obtain  overall  means  for 
each  test  battery.  7)  Finally,  we  used  these  results,  based  on  18 
schools,  to  estimate  the  expected  utility  of  each  candidate  battery 
for  all  military  occupational  specialties  (MOSs) . 

The  first  few  sections  below  describe  the  test  batteries  we 
analyzed  and  the  experimental  design  for  the  study.  The  next  three 
sections.  Derivation  of  Regression  Equations,  Assignment,  and 
Assessment  of  Utility,  address  the  specific  procedures  involved  in 
each  step. 

Test  Batteries 

We  evaluated  the  classification  utility  for  alternative 
combinations  of  individual  tests  from  each  of  the  following 
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existing  batteries:  ASVAB,  CAT-ASVAB,  and  ECAT.  As  indicated  in 
the  introduction,  these  three  batteries  differ  in  terms  of  method 
of  administration  and  abilities  measured.  Table  1  (provided  by 
Segall,  1993)  displays  the  composition  of  the  current  ASVAB 
(denoted  B^)  and  CAT-ASVAB  (denoted  B2)  as  well  as  three 
alternative  batteries  (B3,  B^  and  B5) .  This  table  indicates  the 
ability  measured  and  the  method  of  administration  for  each  test. 
For  example,  the  table  displays  a  "p”  to  indicate  a  paper-and- 
pencil  method  of  test  administration  for  each  test  in  Bj^,  the 
current  ASVAB.  Table  1  also  indicates  other  methods  of 
administration,  including  computer— adaptive  and  computer  non— 
adaptive  (with  and  without  a  response  pedestal) . 


Insert  Table  1  about  here. 


Descriptions  of  the  five  batteries  in  Table  1  follow  (based  on 
descriptions  in  Segall,  1993) : 

1.  ASVAB  (denoted  B^) 

This  is  a  paper-and-pencil  battery,  which  acts  as  a 
baseline  to  determine  the  effect  of  altering  battery 
composition, 

2.  CAT-ASVAB  (denoted  63) 

This  battery  consists  of  the  same  10  tests  as  in  ASVAB, 
but  presented  in  a  computer-adaptive  format.  Two  of  the 
CAT-ASVAB  tests,  NO  and  CS  are  speeded  tests  and  do  not 
employ  an  adaptive  format. 


7 


3. 


ASVAB  +  P&P-ECAT  (denoted  B3) 

This  paper-and-pencil  battery  contains  existing  tests 
plus  those  paper-and-pencil  tests  included  in  the  ECAT 
battery.  Even  though  all  ECAT  tests  are  administered  by 
computer,  several  are  simply  conventional  paper-and- 
pencil  tests  administered  on  a  computerized  platform. 
This  battery  is  intended  to  represent  the  best  paper-and- 
pencil  battery  that  can  be  constructed  from  existing 
tests. 

4.  CAT-ASVAB  +  Non-Pedestal-ECAT  (denoted  B4) 

This  battery  consists  of  CAT-ASVAB  and  ECAT  tests  which 
do  not  use  a  response  pedestal  (Thus  excluding  the  two 
Tracking  tests  and  Target  Identification) .  This  battery 
is  intended  to  represent  the  best  computerized  battery 
that  can  be  constructed  from  available  computerized  non¬ 
response  pedestal  tests. 

5.  CAT-ASVAB  +  All-ECAT  (denoted  B5) 

This  battery  contains  CAT-ASVAB  and  all  tests  included  in 
the  ECAT  battery  (including  those  requiring  a  response 
pedestal) .  It  represents  the  most  comprehensive 
computerized  battery  that  can  be  constructed  from  all 
CAT-ASVAB  and  ECAT  tests. 

The  following  references  contain  descriptions  of  the  tests 
within  each  battery:  ASVAB  Working  Group,  1980;  Wolfe  et  al.,  in 
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press  (ASVAB) ,  Moreno  and  Segall,  1992  (CAT-ASVAB) ,  Wolfe,  et  al., 
in  press  (ECAT) . 

Hypotheses 

The  contract  (#N66001-90-D-9502 ,  DO  7J16)  sow  required  several 
specific  comparisons  among  the  five  test  batteries.  Below,  we 
present  these  specific  comparisons  (slightly  modified  from  the  SOW 
for  clarification) .  In  all  comparisons,  the  total  test  completion 
time  is  optimally  distributed  among  the  individual  tests  to 
maximize  the  differential  validity  of  each  test  combination. 

4.4.1  What  is  the  loss  in  classification 
utility  if  a  shortened  P&P-ASVAB  is  used  in 
place  of  CAT-ASVAB?  (B^  vs  B2) 

Assume  that  the  P&P-ASVAB  completion  time  is 
shortened  to  match  CAT-ASVAB.  There  are 
likely  to  be  gains  in  processing  efficiency 
associated  with  a  shortened  battery.  One 
question  likely  to  arise  is:  Why  not  achieve 
these  gains  by  just  shortening  the  P&P-ASVAB, 
rather  than  implementing  CAT-ASVAB?  This 
comparison  will  demonstrate  the  loss  in 
utility. 

4.4.2  Assuming  that  2.25  hours  (135 
minutes) ,  on  the  average,  are  available  for 
testing,  how  does  the  utility  of  the  best  P&P 
battery  compare  with  that  of  the  best 
computerized  non-pedestal  battery?  (B3  vs  B4) 
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Assiame  that  tests  can  be  selected  from  the 
ECAT  battery  to  supplement  ASVAB  subtests. 
However  there  are  two  restrictions.  First, 
for  the  P&P  battery,  only  those  ECAT  tests 
that  can  be  administered  in  P&P  format  are 
considered  for  inclusion.  Second, 

supplementing  the  computerized  battery,  only 
computerized  non*-pedestal  ECAT  tests  are 
considered  for  inclusion. 

4.4.3  Assuming  that  3  hours  (180  minutes), 
on  the  average,  are  available  for  testing,  how 
does  the  utility  of  the  current  P&P-ASVAB 
compare  with  that  of  the  best  revised  P&P 
battery?  (B^  vs  B3) 

For  the  best  revised  P&P  battery,  only  those 
ECAT  tests  that  can  be  administered  in  P&P 
format  are  considered  for  inclusion.  This 
best  P&P  battery  should  incorporate  optimal 
individual  test  lengths. 

4.4.4  Assuming  that  3  hours  (180  minutes), 
on  the  average,  are  available  for  testing,  how 
does  the  utility  of  the  best  P&P  battery 
compare  with  that  of  the  best  computerized 
non-pedestal  battery?  (B3  vs  B4) 

To  supplement  the  ASVAB  P&P  subtests  and 
create  the  best  P&P  battery,  we  include  only 
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those  ECAT  tests  that  can  be  administered  in 
P&P  format.  For  the  best  computerized  non¬ 
pedestal  battery,  we  supplement  the  CAT-ASVAB 
with  only  those  ECAT  tests  administered 
without  a  response  pedestal.  Both  batteries, 
computerized  and  P&P,  should  incorporate 
optimal  individual  test  lengths. 

4.4.5  Assuming  that  3  hours  (180  minutes) , 
on  the  average,  are  available  for  testing,  how 
does  the  utility  of  the  best  computerized  non¬ 
pedestal  battery  compare  with  that  of  the  best 
computerized  full  battery?  (B4  vs  B5) 

This  comparison  will  assess  the  gain  in 
utility  attributable  to  including  those  tests 
that  require  a  response  pedestal.  Both 
computerized  batteries  should  incorporate 
optimal  individual  test  lengths. 

4.4.6  Assuming  that  3  hours  (180  minutes), 
on  the  average,  are  available  for  testing,  how 
does  the  utility  of  the  best  P&P  battery 
compare  with  that  of  the  best  computerized 
full  battery?  (B3  vs  Bg) 

For  the  best  P&P  battery,  only  those  ECAT 
tests  that  can  be  administered  in  P&P  format 
are  considered  for  inclusion.  For  the  best 
computerized  pedestal  and  non-pedestal 
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battery,  all  ECAT  tests  are  considered 
possible  for  inclusion.  Both  batteries, 
computerized  and  P&P,  should  incorporate 
optimal  individual  test  lengths. 

The  research  questions  above  involve  the  five  test  batteries, 
through  B5,  with  three  of  the  batteries  (B^,  B3,  and  B4) 

evaluated  under  two  different  time  limits.  Since  three  of  the  five 
batteries  require  two  different  completion  time  limits,  there  were, 
in  effect,  eight  candidate  batteries  or  "testing  conditions". 
Table  2  shows  these  eight  testing  conditions  (Aj^  through  Kq)  with 
their  corresponding  test  battery  and  time  limit.  For  example,  in 
this  table,  Aj  involves  a  specific  battery  (B^)  with  a  time  limit 
of  100  minutes. 


Insert  Table  2  about  here. 


The  SOW  requires  six  specific  comparisons,  or  contrasts,  among 
these  candidate  batteries.  Table  3  shows  these  comparisons  by 
their  SOW  task  number.  The  column  "Expected  Completion  Time" 
indicates,  in  minutes,  the  experimental  variation  in  completion 
time  allowed  for  each  of  these  batteries.  For  example,  the  first 
row  of  the  table ,  task  4.4.1,  indicates  a  contrast  between  an  ASVAB 
(Aj^)  and  CAT-ASVAB  (A2)  with  an  expected  completion  time  ec[uivalent 
to  100  minutes. 
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Insert  Table  3  about  here. 


We  evaluated  these  six  contrasts,  or  planned  comparisons, 
among  these  eight  testing  conditions.  Table  4  presents  the  Testing 
Condition  factor  with  eight  levels,  through  A3,  and  the 
coefficients  specifying  the  comparisons.  Although  these  contrasts 
are  not  orthogonal,  they  are  planned  comparisons.  Each  comparison 
in  Table  4  involves  the  difference  between  classification 
efficiency  for  a  specific  pair  of  candidate  batteries.  Because 
increases  in  classification  efficiency  produce  concomitant  gains  in 
classification  utility,  our  hypotheses  below  assume  that,  for  each 
paired  comparison,  the  battery  showing  more  classification 
efficiency  will  also  show  corresponding  gains  in  utility. 


Insert  Table  4  about  here. 


In  Table  4,  a  "-i”  indicates  the  treatment  which  should  have 
a  lower  mean  classification  efficiency.  Likewise,  a  "1"  indicates 
the  treatment  we  expect  to  have  a  higher  mean  efficiency.  The 
hypotheses  for  the  six  planned  comparisons  follow: 

4.4.1  -  Ai  should  have  lower  classification  efficiency  than  Aj. 
Shortening  A^  would  lower  its  reliability,  and  consequently  its 
validity;  and  ultimately,  its  efficiency. 

4.4.2  -  A3  should  have  lower  efficiency  than  A4.  A4  has  all  the 
tests  in  A3,  plus  three  additional  tests.  A4  is  also  a 
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computerized  test.  The  three  additional  tests  and  the  computerized 
mode  of  administration  could  only  increase  classification 
efficiency. 

4.4.3  -  We  expect  A5  to  have  lower  efficiency  than  Ag.  Ag  has  three 
additional  tests,  which  could  only  increase  efficiency. 

4.4.4  -  Ag  should  have  lower  efficiency  than  A7.  As  in  4.4.2,  A7 
is  computerized  and  has  all  the  tests  of  Ag  plus  three  additional 
tests.  The  computerized  mode  and  the  additional  three  tests  could 
only  increase  the  efficiency  of  k-j  over  that  of  Ag. 

4.4.5  -  Here,  we  expect  Aj  to  have  lower  efficiency  than  Ag.  Ag  has 
all  the  tests  of  Ay  plus  three  additional  tests,  which  could  only 
raise  the  efficiency  of  Ag  over  that  of  A7. 

4.4.6  -  Here,  Ag  should  have  lower  efficiency  than  Ag.  Ag  is 
computerized  and  has  all  the  tests  in  Ag  plus  six  additional  tests. 
These  differences  could  only  increase  the  efficiency  of  Ag. 

As  indicated  earlier,  we  estimated  the  classification 
efficiency  and  utility  of  each  battery  using  three  basic  steps.  In 
the  first  step,  we  generated  optimal  regression  equations.  The 
second  step  employed  predicted  performance,  estimated  from  these 
equations,  to  optimally  assign  individuals.  Finally,  the  third 
step  involved  applying  a  utility  index  (SDy)  to  the  assignment 
solutions  and  comparing  the  classification  utility  of  alternate 
test  batteries. 

Experimental  Design 

For  each  of  the  eight  candidate  batteries,  we  created  multiple 
samples  (i.e.,  replications)  for  classification  into  one  of  the 
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study's  assignment  categories.  The  Assignment  section  of  this 
report  describes  those  procedures.  In  the  present  study,  the 
assignment  categories  are  the  18  technical  training  schools 
described  in  Abrahams  et  al.  (1993) .  The  results  from  these  18 
possible  assignments  enabled  us  to  estimate  classification 
efficiency  and  utility  for  each  candidate  battery  (testing 
condition) .  In  addition,  we  extrapolated  the  efficiency  and 
utility  estimates  from  the  limited  number  (18)  of  assignment 
categories  to  the  actual  number  (622)  of  existing  MOSs.  Although 
Brogden  (1959)  demonstrates  that  classification  efficiency 
increases  with  the  number  of  assignments,  his  analysis  is  limited 
to  15  assignment  categories.  To  account  for  the  expected  increase 
in  efficiency  from  classifying  individuals  into  622  MOSs,  we 
developed  a  function  based  on  Brogden 's  procedures  and  assumptions. 
This  function  permits  extrapolation  of  classification  efficiency  to 
several  hundred  assignment  categories.  Before  applying  this 
function,  for  each  candidate  battery,  we  computed  interim  utility 
estimates  for  all  622  assignment  categories.  To  compute  these 
estimates,  we  used  the  mean  predicted  performances  (MPPs)  from  the 
18  schools.  (A  later  section  of  this  report  describes  these 
procedures  in  detail.)  Finally,  we  applied  the  Brogden-based 
adjustment  to  these  interim  utility  estimates  to  compute  our  final 
utility  estimates  for  each  battery. 

The  experimental  design  appropriate  for  evaluating  all  of 
these  comparisons  is  a  randomized  block  design  with  eight  levels  of 
one  treatment  factor  (one  for  each  testing  condition)  .  This  design 
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permits  the  simultaneous  evaluation  of  all  testing  conditions 
(Edwards,  1972). 

Table  5  presents  this  experimental  design,  in  which  A  is  the 
Testing  Condition  Factor  (previously  defined) ,  and  Replications  (R) 
are  the  blocks.  Within  each  replication,  or  block,  exactly  the 
same  individuals  were  available  for  assignment  within  each  of  the 
treatment  conditions.  Edwards  (1972)  discusses  this  design  in  some 
detail  as  well  as  suitable  tests  of  significance.  McNemar  (1962) 
discusses  the  appropriate  significance  test  for  planned  contrasts. 


Insert  Table  5  about  here. 


For  each  of  the  eight  testing  conditions,  we  optimally 
assigned  the  same  100  individuals  in  each  replication.  We  assigned 
each  subject  to  one  of  the  18  schools.  The  assignment  algorithm 
filled  quotas  to  optimize  predicted  performance  for  the  18  schools. 

As  the  subscripts  in  Table  5  illustrate,  we  used  the  same  100 
individuals,  within  each  replication,  for  assignment  in  each  of  the 
testing  conditions.  We  conducted  eight  replications  for  each 
treatment  condition.  The  Assignment  section  describes  the 
procedure  we  used  to  determine  the  size  of  the  simulated  applicant 
samples  and  the  specification  of  the  selection  ratio. 

Derivation  of  Regression  Equations 

Data 

For  each  candidate  test  battery,  we  generated  the  appropriate 
intercorrelation  and  validity  matrix.  These  matrices  include  the 
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relevant  tests  for  each  candidate  battery  and  reflect  the  optimal 
allocation  of  total  test  administration  time  to  each  component 
test.  Ideally,  generating  all  the  necessary  matrices  would  require 
a  sample  of  subjects  who  completed  all  three  batteries  and  were 
then  assigned  to  a  variety  of  treatments  (e.g.,  technical  schools 
or  occupational  specialties).  Unfortunately,  data  were  only 
available  for  the  ASVAB  and  ECAT  batteries  (see  Abrahams  et  al., 

1993) . 

The  basic  data  in  the  Abrahams  et  al.  (1993)  study  are  ASVAB 
and  ECAT  test  scores  and  criteria  for  9,038  enlisted  recruits 
assigned  to  18  technical  schools.  The  criterion  measure  for  most 
of  the  schools  is  Final  School  Grade  (FSG) .  Table  6  presents  a 
listing  of  the  18  schools.  For  a  complete  description  of  all  the 
basic  data,  see  Abrahams  et  al. 


Insert  Table  6  about  here. 


Since  CAT-ASVAB  scores  were  not  available  for  the  18  school 
samples,  we  used  analytical  methods  to  estimate  the  CAT-ASVAB  test 
scores.  We  based  the  estimated  CAT-ASVAB  scores  on  ASVAB  true 
scores,  under  the  assumption  that  corresponding  ASVAB  and  CAT-ASVAB 
tests  measure  the  same  abilities.  Moreno  and  Segall  (Submitted  for 
publication)  support  this  assumption  with  results  showing  a 
virtually  perfect  intercorrelation  between  true  scores  of 
corresponding  tests  in  CAT-ASVAB  and  ASVAB. 
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Corrections  to  ASVAB  and  ECAT  Intercorrelations  and  Validities 

This  section  describes  the  data  matrices  and  procedural  steps 
required  to  generate  corrected  ASVAB  and  ECAT  intercorrelation  and 
validity  matrices. 

Population  #l  Matrix.  This  matrix  is  from  the  1991  DoD 
population  of  applicants  that  Abrahams  et  al.  (1993)  used  in  their 
study.  It  contains  the  means,  standard  deviations,  and 
intercorrelations  for  the  ASVAB  tests  in  the  applicant  population. 
We  refer  to  this  intercorrelation  matrix  as  Population  #l  Matrix. 
This  matrix  is  in  Table  A-1  of  Appendix  A. 

School  Sample  Data  Matrices.  Our  school  sample  data  matrices 
contain  the  range  restricted  intercorrelations  of  ASVAB,  ECAT,  and 
the  criterion,  usually  FSG,  together  with  their  means  and  standard 
deviations,  for  each  of  the  18  school  samples  from  the  Abrahams  et 
al.  (1993)  study.  We  refer  to  these  as  the  School  Sample  nai-,a 
IJatricgs.  Tables  A-2  through  A-19  of  Appendix  A  contain  these  18 
matrices. 

Combined  Schools  Predictor  Matrix.  We  combined  the  ASVAB  and 
ECAT  intercorrelation  data  for  all  18  schools,  yielding  a  combined 
schools  predictor  matrix  of  9038  subjects.  This  matrix  contains 
the  restricted  means,  standard  deviations,  and  intercorrelations 
for  the  10  ASVAB  and  9  ECAT  tests.  We  refer  to  this  matrix  as  the 
Csmbined  Schools  Predictor  Matriv.  This  matrix  is  in  Table  A-20  of 
Appendix  A. 

Population  #2  Matrix.  Using  the  Population  #l  Matrix  as  the 
population  matrix  and  the  Combined  Schools  Predictor  Matrix  as  the 
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restricted  matrix,  we  applied  the  correction  for  multivariate 
restriction  in  range  (Lawley,  1943)  to  estimate  the  population 
intercorrelations,  means,  and  standard  deviations  for  the  ASVAB  and 
ECAT  tests.  The  resulting  matrix  was  a  combined  predictor 
population  matrix,  which  we  refer  to  as  Population  #2  Matrix. 
Table  A-21  of  Appendix  A  contains  this  matrix. 

Corrected  School  Validity  Matrices.  Using  the  Population  #2 
Matrix  as  the  population  matrix,  and  the  18  individual  School 
Sample  Data  Matrices  as  the  restricted  matrices,  we  applied  the 
correction  for  multivariate  restriction  in  range  (Lawley,  1943)  to 
estimate  population  FSG  validities,  plus  means  and  standard 
deviations,  for  each  school.  While  this  correction  also  produced 
predictor  intercorrelations,  as  expected,  they  were  identical  for 
each  of  the  18  schools  and  to  those  in  the  Population  #2  Matrix, 
which  is  in  Table  A-21  of  Appendix  A.  We  refer  to  these  matrices 
as  the  Corrected  School  Validity  Matrices. 

Matrix  of  Fully  Corrected  School  Validities.  As  mentioned  in 
the  preceding  paragraph,  each  of  the  Corrected  School  Validity 
Matrices  for  the  18  schools  were  identical  except  for  the  last 
column  of  corrected  validities,  which  differ  for  each  school.  We 
created  a  new  matrix  from  these  18  columns  of  validities,  making  a 
new  matrix  of  18  rows  and  19  columns.  Each  row  of  the  matrix 
corresponds  to  one  of  the  18  schools,  and  each  column  refers  to  one 
of  the  19  tests.  Thus,  the  ith  row  contains  the  corrected 
validities  for  the  19  tests  in  the  ith  school.  The  validities  in 
this  matrix  were  corrected  for  criterion  unreliability.  We  call 
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this  matrix  the  Matrix  of  Fully  Corrected  School  Validities.  Table 
A-22  of  Appendix  A  contains  this  matrix. 

Of  the  matrices  above,  the  following  three  play  a  fundamental 
role  in  subsequent  analysis: 

1.  Population  #1  Matrix;  Table  A-1,  Appendix  A, 

2.  Population  #2  Matrix;  Table  A-21,  Appendix  A,  and 

3.  Matrix  of  Fully  Corrected  School  Validities;  Table  A-22, 
Appendix  A. 

Reliability  Estimates 

In  addition  to  test  inter correlations  and  validities,  for 
subsequent  analyses,  we  also  needed  test  reliability  estimates  of 
the  three  batteries  for  the  applicant  population.  This  section 
discusses  how  we  used  the  corrected  matrices  (Population  Matrix  #1 
and  #2)  to  estimate  these  reliabilities  for  the  applicant 
population. 

A8VAB  Reliabilities.  We  obtained  reliability  data  from  a 
study  (Moreno  &  Segall,  submitted  for  publication)  in  which  a  group 
of  Navy  recruits  completed  two  non-operational  forms  of  the  ASVAB. 
Each  recruit  in  the  sample  also  completed  the  operational  ASVAB 
prior  to  enlistment.  The  intercorrelations  among  the  three  ASVAB 
forms,  together  with  their  means  and  standard  deviations, 
constitute  the  ASVAB  Reliability  Matrix. 

We  used  the  Population  #1  Matrix  as  the  unrestricted  matrix 
and  the  ASVAB  Reliability  Matrix  as  the  restricted  matrix.  We 
applied  the  correction  for  multivariate  restriction  in  range  to 
yield  the  population  means,  standard  deviations  and 
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intercorrelations  for  the  operational  ASVAB  (the  explicit  selection 
variables)  and  the  two  non-operational  ASVAB  forms  (the  implicit 
selection  variables)  .  This  correction  yielded  the  ASVAB  Corrected 
Reliability  Matrix. 

Originally,  we  planned  to  use  the  correlation  between 
corresponding  tests  on  the  two  non-operational  ASVAB  forms  to 
represent  the  subtest  reliabilities.  However,  preliminary  analysis 
revealed  the  second  subtest  reliabilities  were  too  low,  having  been 
degraded  by  various  factors  (see  Moreno  &  Segall,  submitted  for 
publication) .  Relying  on  the  structural  analysis  of  Moreno  and 
Segall,  we  used  the  reliability  of  the  first-administered  non- 
operatipnal  ASVAB  form  to  estimate  the  subtest  reliabilities  (i.e.  ,- 
ASVAB  Form  9B)  .  Table  4  in  Moreno  and  Segall  presents  these 
reliabilities.  Using  the  ASVAB  Form  9B  reliabilities,  together 
with  the  subtest  standard  deviations  from  the  ASVAB  Reliability 
Matrix  and  the  ASVAB  Corrected  Reliability  Matrix,  we  estimated  the 
population  subtest  reliabilities  using  the  following  equation  from 
Kelley  (1921) .  This  equation  assumes  equality  of  error  variance  in 
the  population  and  the  sample; 

where  the  capital  R  and  S  refer  to  the  Population  02  Matrix 
reliability  and  standard  deviation,  respectively,  and  the  lower 
case  r  and  s  refer  to  the  sample  restricted  reliability  and 
standard  deviation,  respectively,  for  test  i.  Table  B— 1  in 
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Appendix  B  presents  the  restricted  and  unrestricted  standard 
deviations  and  reliabilities. 

SCAT  Reliabilities.  Larson  and  Alderton  (1992)  conducted  a 
test-retest  study  of  the  reliability  of  the  ECAT.  The  nine  ECAT 
tests  were  administered  twice  to  a  sample  of  313  High  School  and 
Junior  College  students,  with  a  retest  interval  of  four  to  five 
weeks.  Larson  and  Alderton  provide  the  standard  deviations  and 
reliabilities  of  the  nine  ECAT  tests  in  this  sample.  We  corrected 
these  reliabilities  in  the  student  sample  for  restriction  in  range 
by  substituting  the  estimated  population  ECAT  standard  deviations 
from  our  Population  #2  Matrix,  together  with  the  ECAT  sample 
standard  deviations  and  reliabilities,  into  Kelley's  (1921) 
equation,  presented  above  as  our  equation  (1)  .  Table  B-2  in 
Appendix  B  presents  the  sample  and  population  ECAT  standard 
deviations  and  test-retest  reliabilities. 

CAT-A8VAB  Reliabilities.  The  eight  power  tests  in  the  CAT- 
ASVAB  are  administered  in  a  tailored  testing  manner,  so  that  each 
successive  item  is  chosen  to  be  most  appropriate  to  the  examinee's 
estimated  ability,  based  on  their  performance  on  previous  items. 
As  a  result,  the  gain  in  reliability  expected  from  each  successive 
item  is  greater  than  would  be  expected  using  ordinary  paper-and- 
pencil  methods.  In  this  way,  the  CAT-ASVAB  power  tests  are  able  to 
achieve  greater  reliability  with  fewer  items  and  in  less  time  than 
their  paper-and -pencil  ASVAB  counterparts.  However,  because  of  the 
adaptive  nature  of  the  testing,  the  conventional  Spearman-Brown 
formulas  are  not  appropriate  for  estimating  gains  in  reliability 
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due  to  lengthening  the  tests.  Consequently,  the  CAT-ASVAB  requires 
a  reliability  function  appropriate  for  adaptive  tests. 

Segall  (1993)  developed  an  equation  for  estimating  the  CAT- 
ASVAB  reliability  function  for  the  power  tests.  This  equation  is: 


Pil*Ut^)  =  l-El+UjCn^^  X 


JL_ 

I'i 


(2) 


in  which: 

Pii(4)(t^)  is  the  reliability  of  the  ith  test  at  its  revised 
expected  completion  time, 

t^  is  the  expected  completion  time  of  the  ith  test, 
no(i)  number  of  items  in  the  original  ith  test, 

is  the  expected  completion  time  of  the  original  ith  test, 

and 

Ui,  v^,  and  w^  are  the  reliability  functions  parameters  for 
test  i. 

Table  C-1  presents  the  above  constants  (derived  by  Segall, 
1994)  for  each  of  the  CAT-ASVAB  power  tests. 

Auto/Shop  Information  (AS)  Reliabilities.  While  the  ASVAB 
measures  Auto/ Shop  Information  with  a  single  subtest,  AS,  the  CAT- 
ASVAB  measures  Auto  Information  (AI)  and  Shop  Information  (SI) 
separately.  Segall  (1994)  combined  the  AI  and  SI  tests  m  a  way  to 
be  comparable  to  the  ASVAB  AS  subtest,  and  provided  the  constants 
to  enter  this  combination  into  his  equation  5.11  (our  equation  (2) 
above) .  Table  C-1  includes  these  constants. 
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Estimating  CAT-ASVAB  Intercorrelations 

As  indicated  earlier,  CAT-ASVAB  test  scores  were  not  available 
for  our  samples.  Consequently,  we  estimated  CAT-ASVAB 

intercorrelations  and  validities  from  the  ASVAB  data  using  the 
procedures  in  Segall  (1993).  Below,  we  describe  these  procedures, 
which  require  ASVAB  intercorrelations  and  reliabilities  and  CAT- 
ASVAB  reliabilities. 

The  Population  #2  Matrix  contains  the  population 

intercorrelations  for  the  paper-and-pencil  ASVAB.  We  obtained  the 
reliabilities  of  the  P&P-ASVAB  subtests  from  the  ASVAB  Corrected 
Reliability  Matrix,  as  described  earlier.  We  refer  to  the  ith  P&P- 
ASVAB  test  reliability  as 

We  determined  the  CAT-ASVAB  reliabilities  at  their  original 
lengths  from  the  reliability  functions  provided  by  Segall  (1994) . 
We  refer  to  the  reliability  of  the  ith  CAT-ASVAB  subtest  as 

We  derived  a  weight  for  each  ASVAB  test,  which  is  the  square 
root  of  the  ratio  of  the  two  reliabilities: 


We  obtained  the  CAT-ASVAB  intercorrelations  from  the  P&P-ASVAB 
intercorrelations  as  follows: 

Tcd.j)  -  rp(i,j)  («i)  (Wj)- 

The  CAT-ASVAB  reliabilities  at  their  original  lengths  can  be 
obtained  by  entering  the  original  expected  completion  times  for  the 
tests  in  equation  (2) .  Table  C-2  presents  the  reliabilities  and 
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weights  for  the  eight  CAT-ASVAB  power  tests.  Table  C-3  shows  the 
estimated  CAT-ASVAB  intercorrelations . 

Estimating  CAT-ASVAB  Validities 

The  Matrix  of  Fully  Corrected  School  Validities  contains  the 
population  validities  for  the  i  =  10  P&P— ASVAB  tests  in  each  of 
k  =  18  schools.  We  obtained  the  population  validities  for  the 
i  =  10  CAT-ASVAB  counterpart  tests  using  the  equation: 

^C(i,k)  =  (Wi). 

where  the  w^  values  are  the  same  as  those  defined  in  the  section 
above.  Table  C-4  presents  these  estimated  CAT-ASVAB  validities. 

Parenthetically,  the  same  general  procedure  can  be  used  to 
estimate  the  intercorrelations  and  validities  of  any  battery,  where 
the  W^  values  are  defined  as: 

w,  = 

Individual  Test  completion  Times 

Subsequent  analyses  involving  optimal  allocation  of  total 
battery  time  required  completion  times  for  the  individual  tests  in 
each  battery.  Table  D-1  in  Appendix  D  presents  these  times.  The 
following  sections  describe  the  derivations  of  the  table  entries. 

ASVAB.  Because  of  the  lock-step  nature  of  the  P&P-ASVAB,  the 
test  time  is  the  same  for  everyone  and  corresponds  to  the  times 
allotted  to  each  of  the  tests.  Table  1  in  Wolfe,  Alderton,  and 
Larson  (in  review),  provides  these  time  limits. 


new  ith  test  reliability 
original  ith  test  reliability 
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SCAT.  For  computer  administered  tests,  expected  completion 
time  is  taken  as  the  average  completion  time  among  examinees.  The 
Abrahams  et  al.  (1993)  study  provided  estimates  of  the  mean  ECAT 
test  times.  Using  Lawley's  (1943)  multivariate  correction,  we 
obtained  estimates  of  applicants'  ECAT  mean  test  times  based  on  the 
means  observed  in  the  school  sample. 

Three  of  the  ECAT  subtests,  FR,  AO,  and  SO,  can  be 
administered  directly  in  paper-and -pencil  form.  Using  data  from 
the  Abrahams  et  al.  (1993)  study,  we  created  logarithms  of  the  FR, 
AO,  and  SO  subtest  times.  We  used  these  logarithms,  together  with 
the  selectee  ASVAB  scores,  to  construct  the  sample  matrix,  and  we 
used  the  Population  #1  Matrix  as  the  population  matrix.  We  applied 
Lawley's  (1943)  procedure  to  these  matrices  to  obtain  estimated 
population  means  and  standard  deviations  for  testing  times.  We 
then  converted  the  95th  percentile  of  the  log  times  to  actual 
times.  We  used  these  times  as  the  time  limits  for  the  paper  and 
pencil  FR,  AO,  and  SO  ECAT  subtests.  Table  D-1  Presents  completion 
times  for  all  ECAT  tests. 

CAT-A8VAB.  Because  CAT-ASVAB  is  an  adaptive  battery, 
individuals  can  proceed  at  their  own  pace,  avoiding  the  lock-step 
requirement  of  the  same  testing  time  for  everyone.  Consequently, 
we  used  the  average  testing  times  (presented  in  Segall,  1993,  Table 
5.8)  to  determine  the  required  testing  time  for  each  subtest. 
Segall  provided  the  population  expected  completion  times  for  the 
CAT-ASVAB.  Table  D-1  in  Appendix  D  lists  these  completion  times. 
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optimal  Allocation  of  Total  Battery  Time 

Recall  that  all  of  the  battery  comparisons  require  a  method  to 
determine  the  optimal  allocation  of  test  time  to  individual  tests 
that  maximize  differential  validity.  Though  several  methods  have 
been  proposed  for  selecting  tests  for  inclusion  in  a  differential 
aptitude  battery,  only  Horst  and  MacEwan  (1957)  provide  an 
analytical  method  for  determining  optimal  allocation  of  total  test 
administration  time  to  the  individual  tests  in  a  battery  while 
maximizing  differential  validity.  Extensive  studies  by  Johnson  et 
al.  (1990)  failed  to  reveal  any  procedure  that  resulted  in 
assignments  superior  to  those  derived  using  Horst's  (1956)  methods. 
For  this  reason,  we  employed  Horst's  procedures. 

Essentially,  this  procedure  (Horst,  1956)  uses  population- 
corrected  validity  and  reliability  information  to  simultaneously 
modify  all  subtest  lengths  for  a  fixed  total  time,  with  the 
objective  of  identifying  the  set  of  test  lengths  that  maximize 
differential  prediction  across  assignments.  Since  this  is  an 
iterative  procedure,  it  is  more  efficient  if  the  time  limit  of  the 
battery  for  the  initial  matrix  is  approximately  equal  to  the 
desired  total  time  at  the  beginning  of  the  iterative  process.  We 
used  Horst's  (1956)  Phi  index  to  measure  the  magnitude  of 
differential  prediction  at  each  iteration. 

To  implement  this  process,  we  divided  Horst's  (1956)  procedure 
into  two  programs.  The  first,  HORST-Ll . BAS ,  generates  an  initial 
matrix  of  validities  and  intercorrelations  based  on  a  constant 
proportionate  reduction  in  the  length  of  each  test  as  a  first 
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approximation.  Appendix  E  illustrates  the  procedure  for 
proportionate  reduction  to  .8  of  original  test  time.  The  second 
program,  H0RST-L2 . BAS ,  then  uses  these  data  to  generate 
approximations  2  through  20.  We  thought  this  would  be  a  sufficient 
number  of  iterations  for  just  about  any  problem.  In  one  instance, 
when  we  needed  more  iterations,  we  used  the  validities  and 
intercorrelations  from  the  20th  iteration  as  program  input  to 
generate  a  second  set  of  20  iterations.  This  process  could  be 
repeated  any  number  of  times.  However,  after  40  iterations,  the 
test  lengths  had  fully  stabilized. 

We  first  applied  the  Horst  (1956)  procedure  to  the  P&P-ASVAB 
with  100  minutes  testing  time,  and  then  to  the  P&P-ASVAB  with  180 
minutes  testing  time.  Tables  F-1  and  F-2  of  Appendix  F  present  the 
optimal  test  times,  intercorrelations,  and  validities  for  the  P&P- 
ASVAB  with  100  minutes  total  testing  time.  Tables  F-3  and  F-4  of 
Appendix  F  present  these  same  statistics  for  the  P&P-ASVAB  with  180 
minutes  total  testing  time. 

Horst's  (1956)  method  for  determining  optimal  test  length  for 
differential  prediction  is  based  on  the  relation  between  error 
variance  and  test  length  when  the  Spearman-Brown  formula  (Guilford, 
1965)  is  appropriate.  With  Computer  Adaptive  Tests,  the  Spearman- 
Brown  formula  is  not  appropriate;  so  direct  application  of  Horst's 
procedure  will  yield  incorrect  results.  Therefore,  we  modified  the 
Horst  procedure  to  incorporate  the  non-linear  reliability  functions 
described  earlier  for  the  CAT-ASVAB. 
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We  then  applied  the  Horst  (1956)  procedure  to  the  remaining 
testing  conditions.  Table  7  provides  the  resulting  optimal  test 
times  for  each  of  these  eight  conditions.  Appendix  F  (Tables  F-5 
through  F-20)  contains  the  intercorrelations  and  validity  matrices, 
corresponding  to  the  optimal  test  times,  for  all  eight  testing 
conditions.  We  used  these  matrices  to  generate  standard  regression 
equations  for  predicting  school  performance. 


Insert  Table  7  about  here. 


Table  8  presents  the  original  test  lengths  and  the 
proportionately  increased  test  lengths  for  a  100-minute  CAT-ASVAB 
battery.  One  can  compare  the  test  times  in  Table  8  with  those  in 
column  A2  of  Table  7  to  see  the  effects  of  the  Horst  (1956) 
procedure  on  optimal  allocation  of  test  time.  The  results  for  this 
CAT-ASVAB  battery  show  WK  and  El  as  the  two  tests  with  the  largest 
proportional  increases  in  test  time.  Furthermore,  these  tables 
show  AR  and  CS  as  the  two  tests  yielding  the  greatest  decrease. 


Insert  Table  8  about  here. 


Assignment 

Deriving  Simulated  Test  Scores  and  Expected  criterion  Scores 
The  research  design  we  used  required  replication  on  several 
samples.  To  create  multiple  samples  with  test  scores  on  all  three 
test  batteries,  we  created  synthetic  subjects  randomly  drawn  from 
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a  multivariate  normal  distribution.  We  created  this  multivariate 
normal  distribution  from  the  Population  #2  Matrix,  after  correcting 
for  attenuation. 


Insert  Table  9  about  here. 


We  corrected  only  19  of  the  29  tests  (i.e.,  10  ASVAB,  and  9 
ECAT  tests)  for  attenuation.  (The  ten  CAT-ASVAB  scores  were 
derived  from  the  simulated  ASVAB  scores  generated  by  the  procedures 
described  in  this  section.)  Table  9  presents  the  resulting  true 
score  intercorrelation  matrix.  We  factor  analyzed  this  true  score 
matrix  using  the  principal  components  method.  We  used  the  first  16 
of  the  18  factors  shown  in  Table  G-1  of  Appendix  G.  Table  G-2  of 
Appendix  G  presents  the  factor  loadings  for  the  19  tests.  The 
factor  analysis  yielded  a  set  of  16  factor  loadings  for  each  test 
in  the  battery.  For  any  simulated  individual,  we  drew  16  random 
normal  deviates,  standing  for  the  16  factors,  respectively.  Each 
individual's  simulated  score  on  any  test  is  the  sum  of  the  products 
of  the  normal  deviates  multiplied  by  their  respective  factor 
loadings  for  that  test  (Guilford,  1965) .  We  refer  to  these  test 
scores  as  the  individual's  "true  scores",  since  we  statistically 
eliminated  all  error  variance.  These  "true  scores",  which  we 
attenuated  by  the  appropriate  reliability,  formed  the  basis  for 
generating  "observed  scores"  for  any  specific  battery  in  a  contrast 
of  interest . 
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To  generate  "observed  scores"  for  the  tests  in  any  specific 
battery,  we  drew  one  additional  random  normal  deviate  for  each  test 
in  the  battery  to  represent  the  test's  "error"  component.  As  the 
following  formula  illustrates,  we  computed  the  "observed  score" 
(Xq)  for  any  test  by  multiplying  the  previously  computed  "true 
score"  (X|.)  by  the  square  root  of  the  test's  reliability 
the  "error  component"  (©x)  random  normal  deviate  for  that  test, 
multiplied  by  the  square  root  of  one  minus  the  test's  reliability 
at  the  test's  optimal  length. 

~  ~^xx 

similarly,  we  created  CAT-ASVAB  scores  from  the  corresponding 
ASVAB  true  scores  and  the  CAT-ASVAB  reliabilities  using  the  same 
equation.  Then,  we  computed  the  expected  criterion  scores  for  each 
individual  by  applying  the  previously  developed  least-squares 
regression  equations  to  the  individual's  "observed  scores".  These 
scores  are  Z  scores  (mean  =  0,  S.D.  —  1),  standardized  using  the 
1991  applicant  data. 

Using  this  method,  we  created  eight  applicant  samples,  each 
containing  100  subjects.  From  each  applicant  sample,  we  assigned 
that  number  of  applicants  to  entry  level  schools  to  correspond  to 
the  proportion  of  1992  applicants  assigned  to  entry  level  schools. 
We  set  the  individual  school  quotas  to  match  the  actual  relative 
proportions  assigned  to  the  18  schools  in  FY92 .  Table  10  contains 
the  estimated  school  quotas  we  used  for  assignment  simulations. 
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Insert  Table  10  about  here. 


Estimating  Individual  Marginal  Utility  in  Dollars 

The  regression  equations  described  in  the  previous  sections 
yield  predicted  performance  (PP)  scores  for  individuals.  These 
scores  alone  do  not  reflect  the  dollar  value  of  various  levels  of 
performance  in  various  assignments.  At  this  point,  the  original 
research  plan  called  for  developing  estimates  of  individual 
marginal  utility  in  dollars.  We  planned  to  estimate  individual 
level  utility  by  considering  "per  person"  dollar  value  within  each 
job.  After  collecting  financial  data  reflecting  the  dollar  value' 
of  various  military  jobs,  we  found  evidence  that  the  economic 
values  observed  for  military  jobs  may  not  adequately  reflect  their 
perceived  value  to  the  military.  The  rest  of  this  section  presents 
our  observations  and  explains  why  we  altered  our  research  plan  to 
enter  only  the  PP  scores  into  the  assignment  algorithm. 

Based  on  techniques  described  in  a  later  section.  Table  11 
presents  an  estimated  value  called  "dollar  job  value"  for  17  of  the 
18  occupational  specialties  we  included  in  this  study.  Our 
validity  study  investigated  two  training  subspecialties  within  the 
Army  IIH  MOS,  but  the  DMDC  financial  data  could  provide  only  one 
job  value  for  both  subspecialties.  Consequently,  we  have  only  17 
unique  dollar  job  values.  (Please  see  the  section  entitled 
"Assessment  of  Utility"  for  an  explanation  of  how  we  computed  the 
dollar  value  of  jobs.)  Private  sector  salary  administrators  might 
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expect  that  a  job  in  a  higher  complexity  category  should  have  two 
or  three  times  the  value  of  a  job  in  a  lower  complexity  category. 
(A  later  section  of  this  report  presents  evidence  supporting  this 
expectation.)  The  data  from  our  sample,  however,  show  that  the 
average  ($130,921.86)  medium  complexity  jobs  show  only  about  25% 
more  value  than  average  ($104,906.60)  for  low  complexity  jobs. 
Also,  practically  no  difference  exists  (and  not  in  the  predicted 
direction)  on  mean  job  value  between  the  two  highest  job  complexity 
categories . 


Insert  Table  11  about  here. 


The  lack  of  meaningful  differences  in  the  dollar  job  values  by 
complexity  category  suggests  that  financial  data  alone  might  not 
adequately  reflect  job  value  for  military  jobs.  To  examine  this 
possibility,  we  used  data  contained  in  Bobko  and  Donnelly  (1988). 
They  used  a  ratio  scale  to  collect  utility  ratings  from  field  grade 
Army  officers  for  19  Army  MOSs.  Using  the  financial  data  we 
collected,  we  located  15  of  these  19  MOSs  in  our  database  and 
computed  their  economic—based  30b  values.  Then,  we  correlated 
those  job  values  with  the  ratings  in  Bobko  and  Donnelly  for  the 
50th  performance  percentile.  The  resulting  correlation  (-.196)  was 
not  in  the  predicted  direction  and  not  significantly  different  from 
zero. 

Table  12  presents  further  information  supporting  the  disparity 
between  military  and  civilian  pay  by  job  complexity  category.  To 
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create  Table  12,  we  used  data  from  Hunter,  Schmidt,  and  Judiesch 
(1990)  to  identify  several  civilian  jobs  in  the  same  job  complexity 
categories  as  our  military  jobs.  For  the  civilian  jobs  we 
identified  in  this  way,  we  obtained  mean  salaries  from  a  local 
(Southern  California)  salary  survey.  Our  Table  12  presents  mean 
monthly  salaries  by  job  and  by  job  complexity  category  for  the 
military  and  civilian  jobs.  We  computed  the  military  mean  salaries 
from  data  provided  by  DMDC. 


Insert  Table  12  about  here. 


Inspection  of  Table  12  shows  that  average  salaries  for 
civilians  in  the  lowest  category  equal  about  98%  of  the  military 
salaries  in  that  category.  The  similarity  in  pay  structure, 
however,  appears  to  stop  there.  The  average  civilian  salary  in  the 
middle  complexity  category  is  about  94%  greater  than  the  salary  of 
the  lowest  complexity  category,  while  the  average  military  salary 
is  only  about  25%  larger.  Looking  at  the  top  two  civilian 
categories,  we  find  that  average  salaries  in  the  highest  category 
are  about  2.7  times  the  average  salaries  in  the  middle  category. 
Looking  at  the  corresponding  categories  for  two  military  jobs,  we 
find  that  average  salaries  in  the  first  category  are  slightly  less 
than  the  average  salaries  in  the  middle  category. 

These  findings  support  the  contention  that  the  military  places 
value  on  their  jobs  independent  of  economic  value  reflected  by  what 
it  pays  for  those  jobs.  Recognizing  this,  we  decided  NOT  to  base 


34 


assignments  on  predicted  utility  but  instead  on  predicted 
performance.  By  basing  assignments  only  on  predicted  performance, 
we  could  still  estimate  utilities  after  optimal  assignment  from 
increments  in  predicted  performance.  In  addition,  others  could 
apply  alternative  measures  of  utility  to  these  same  mean 
performance  estimates  available  from  each  candidate  battery.  By 
excluding  utility  from  the  assignments,  we  exercised  care  NOT  to 
over  emphasize  the  importance  of  the  dollar  value  of  jobs  to  the 
military. 

Next,  we  compared  the  job  values  by  complexity  category  for 
the  16  MOSS  in  Bobko  and  Donnelly  (1988).  Table  13  presents  these 
data  in  the  same  format  we  used  to  show  such  data  from  this  study- 
in  Table  11.  The  mean  job  value  for  complexity  category  2  is 
higher  than  the  value  for  category  3  in  both  studies  (about  25% 
higher  in  this  study  and  about  17%  higher  in  Bobko  and  Donnelly) . 
This  offers  some  support  that  the  dollar  values  we  plan  to  use 
later  in  this  study  reflect  the  expected  relationship  to 
complexity. 

We  point  out,  however,  that  the  combination  of  these  findings 
suggests  the  need  to  eventually  develop  a  concept  of  value  for 
military  jobs  that  considers  factors  beyond  dollar  value.  For  this 
study,  then,  these  economic  job  values  will  likely  underestimate 
the  true  job  value  as  perceived  by  the  military.  Therefore,  these 
economic  job  values  will  likely  provide  conservative  estimates  and 
underestimate  the  value  of  whatever  gains  are  realized  from  an 
improved  selection  system. 
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Insert  Table  13  about  here. 


Based  on  our  findings  regarding  the  economic  value  of  jobs  in 
the  military,  we  used  only  PP  scores  as  input  for  the  assignment 
procedure.  (We  described  the  procedures  to  compute  the  PP  scores 
the  section  entitled  "Derivation  of  Regression 
Eguatiohs" . )  For  each  case,  we  computed  the  scores  for  every 
school  in  our  sample  as  a  matrix  of  PP.  This  matrix  served  as  the 
input  for  the  assignment  procedure  described  next. 

Assignment  Algorithm 

We  used  software  provided  by  NPRDC  to  optimally  assign 
individuals  to  schools.  This  software  is  based  on  the  Ford- 
Fulkerson  transportation  algorithm  (Ford  &  Fulkerson,  1956) .  Using 
the  PP  scores  described  above,  this  software  assigned  individuals 
to  maximize  the  sum  of  the  PP  values  across  individuals'  school 
assignments.  Our  SOW  does  not  require  us  to  incorporate  policies 
included  in  the  military's  operational  assignment  system. 
Incorporating  such  policies  might  increase  the  difficulty  of 
interpreting  the  results  of  our  comparisons.  We  assume  their 
exclusion  did  not  affect  the  relative  difference  between  batteries. 

The  input  to  the  program  included:  the  number  of  assignment 
categories  (i.e.,  schools),  the  total  number  of  individuals  to  be 
assigned,  the  quota  for  each  school,  and  a  predicted  performance 
score  for  each  school  for  every  person.  For  each  contrast,  we 
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applied  the  Ford-Fulkerson  procedure  to  assign  eight  samples  of 
individuals  to  schools. 

An  additional  concern  in  generalizing  from  our  sample  of  18 
schools  to  all  military  jobs  is  the  impact  of  the  number  of 
possible  assignments  on  classification  utility.  As  Brogden  (1959) 
indicates : 

The  nximber  of  jobs  is  highly  important  to  efficiency  of 
classification.  With  other  factors  constant,  the 
allocation  average  will  double  in  going  from  2  to  5  jobs 
and  triple  in  going  from  2  to  13  jobs.  (p.  189) 

Although  the  military  assigns  enlisted  personnel  to  over  500  entry- 

level  occupational  specialties,  the  data  for  the  present  study  was 

limited  to  18  specialties.  This  limitation  required  extrapolation 

of  the  classification  utility  observed  for  the  18  specialties  to 

the  utility  for  all  specialties.  Using  the  results  of  varying  the 

number  of  possible  assignments  (see  Table  5) ,  we  generated  a 

function  on  which  to  base  this  extrapolation. 

Assessment  of  Utility 

Hypothesis  testing  during  this  project  compared  the  effect  of 
using  one  selection  battery  or  approach  versus  another  on  the 
resulting  mean  PP  scores.  Where  hypothesis  testing  produced 
significant  differences,  we  used  the  procedures  below  to  assess  the 
dollar  utility  of  those  differences. 

Brogden  (1946,  1949)  developed  the  basic  formula  for  the 

marginal  utility  per  selectee  (i.e.,  the  mean): 
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or. 


A  U,  =  r^SDyZ^^ 


(1) 


where : 

Yg  is  the  mean  predicted  value  of  individual  performance  on 
the  job  expressed  in  dollars. 

My  (mean  output)  is  the  population  mean  of  individual  job 
performance  in  dollars  (or  for  individuals  selected 
randomly) , 

^Ug  is  the  mean  marginal  utility  per  selectee, 
rjjy  is  the  population  correlation  between  ability  and 
performance  (for  the  applicant  population) , 

SDy  is  the  standard  deviation  of  job  performance  in  dollars 
in  the  applicant  population,  and 

is  the  mean  predictor  score  expressed  in  standard  score 
units  (mean  0,  standard  deviation  l)  of  those  selected. 
It  follows  that  the  marginal  utility  for  a  particular  selectee 
(AUg)  is  given  by: 


^  ^8  ~ 


(2) 


Finally,  the  total  marginal  utility  (Au^^^gj^)  is  the  mean  gain 
per  selectee  from  equation  (1)  times  the  number  of  people  selected. 
Ns: 

=  Ns^xySDyZ^^  (3) 


38 


or, 

Wlotal  =  N,  4U,  «) 

Notice  that  formulas  (3)  and  (4)  resemble  those  for  computing 
a  sum  of  X  based  on  knowing  the  mean  X  score  and  the  total  N  (i«e. , 
S  X  =  N  *  3?  ) .  Just  as  the  S  X  represents  the  accumulated  value 
of  all  X  scores,  the  AU^^tal  accumulates  marginal  utilities  over  all 
selectees  (AUg) .  In  each  case,  the  formulas  accomplish  this  by 
multiplying  the  mean  value  by  the  number  of  observations 
accumulated. 

Also  notice  that  the  formulas  presented  above  include  the 
value  of  the  incremental  performance  above  baseline  (random 
selection) ,  but  do  not  include  costs  such  as  those  associated  with 
testing.  In  their  research,  Schmidt  et  al.  (1987)  ignore  the  cost 
of  testing  as  "negligible  relative  to  utility  gains".  The  purpose 
of  this  study  is  different  in  that  it  compares  the  utility  of 
various  batteries.  Therefore,  we  considered  reducing  AU  by  the 
cost  of  testing.  Also,  utility  analysts  (e.g.,  Cascio,  1989; 
Boudreau  &  Berger,  1985)  typically  prefer  to  include  terms  in  the 
equation  to  adjust  AU^-gtai  other  costs  such  as  those  for 

attrition,  recruiting,  or  processing  efficiency.  Later  in  this 
section,  we  explain  why  we  did  not  need  to  adjust  AU  for  such  costs 
in  this  study.  At  that  time,  we  will  revisit  these  issues  together 
with  the  issues  of  net  present  value. 

Essentially,  equation  (2)  describes  the  components  of  the  AUg 
term  for  each  selectee.  Previous  sections  described  how  we 
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obtained  predicted  criterion  scores  (or  predicted  performance,  PP, 
scores)  for  use  in  assignment.  These  scores  are  conceptually 
equivalent  to  the  product  of  and  Zj^,  two  of  the  components  in 
equation  (2) .  In  the  first  subsection  below,  we  describe  the 
procedures  we  used  to  compute  or  estimate  the  remaining  component, 
SDy,  for  each  job. 

Estimating  SDy 

To  estimate  SDy,  we  made  use  of  the  SDp  ratio.  This  (SDp)  is 
the  estimated  performance  at  one  standard  deviation  from  the  mean 
divided  by  the  estimated  performance  at  the  mean.  Judiesch, 
Schmidt,  and  Mount  (1992)  describe  an  objective  method  for 
estimating  the  value  of  average  employee  output.  They  conclude 
that  the  product  of  this  value  and  the  mean  supervisory  ratings  of 
SDp  yields  an  unbiased  estimate  of  SDy. 

A  common  criticism  of  utility  work,  however,  is  that  estimates 
(especially  of  SDy)  tend  to  rely  on  judgmental  rather  than 
objective  data.  To  avoid  this  criticism,  we  chose  to  use  only 
objective  data  in  our  basic  analyses.  (A  follow-up  study  may  check 
the  impact  of  using  judgmental  data  through  sensitivity  analyses.) 
with  objective  estimates  throughout  this  process,  we  avoided 
judgmental  error  in  computing  SDy  values  for  each  enlisted  job. 

Hunter,  et  al.  (1990)  reviewed  studies  between  1937  and  1987. 
In  their  review,  they  selected  studies  that  reported  either  SDp  or 
objective  data  from  which  they  could  calculate  SDp.  Particularly 
important  is  that  their  review  included  only  studies  reporting  on- 
the-job  output  or  studies  using  work  sample  measures  based  on  ratio 
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scales  of  output.  They  included  only  those  studies  which  use  a 
count  of  output  to  compute  a  score  (either  total  or  acceptable 
output) .  They  specifically  excluded  studies  having  job  sample 
measures  based  on  ratings  of  quality  or  quantity  of  output. 

Hunter  et  al.  (1990)  identified  59  jobs  from  studies  meeting 
their  criteria.  First  they  calculated  the  observed  incumbent  SDp 
values;  then  they  corrected  those  for  unreliability;  and  finally 
they  corrected  them  for  range  restriction.  With  these  procedures, 
they  computed  applicant  SDp  values.  If  we  ignore  the  sales  jobs  in 
their  review,  they  report  the  following  average  SDp  values  (as 
percentages  of  mean  output)  by  job  complexity:  low  complexity  (40 
jobs)  —  19.3%,  medium  complexity  (12  jobs)  —  31.8%,  and  high 
complexity  (7  jobs)  --  47.5%. 

[In  a  later  section  where  we  recommend  sensitivity  analyses 
for  a  later  study,  we  present  results  of  SDp  ratings  on  one  school 
in  each  of  the  complexity  categories.  Funding  for  this  project 
prevented  us  from  obtaining  larger  amounts  of  data  and  from 
conducting  sensitivity  analyses.  Therefore,  another  reason  to  use 
the  SDp  values  provided  by  the  literature  is  that  they  are  based  on 
more  jobs  (i.e.,  59  jobs  from  the  literature  versus  3  or  4  jobs  in 
this  study) . ] 

In  addition  to  these  measures  of  SDp,  we  needed  objective 
estimates  of  the  mean  value  of  employee  output  per  job  (Y)  to 
compute  objective  measures  of  SDy  (i.e.,  SDp  *  Y) .  Judiesch  et  al. 
(1992)  provide  an  objective  procedure  for  computing  Y.  In 
developing  their  procedure,  they  begin  by  pointing  out  that  the 
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mean  value  of  employee  output  (Y)  is  equal  to  total  sales  revenue 
divided  by  the  total  number  of  employees.  They  add  that  the 
revenue  value  of  output  will  generally  include  contributions  from 
more  than  one  job  within  an  organization.  Under  these  conditions, 
they  argue  that  an  approximate  estimate  of  the  average  revenue 
value  for  a  particular  job  can  be  calculated  under  the  assumption 
that  the  contribution  of  each  job  to  the  total  revenue  of  an 
organization  is  proportional  to  its  share  of  the  organization's 
total  annual  payroll.  They  present  the  following  formulas  (their 
formula  numbers  in  brackets)  for  a  specific  job  (A) : 

Job  A  Value  =  total  revenue  *  (Job  A  payroll/total  payroll)  [31] 
Y  =  Job  A  Value  /  Job  A  number  of  employees  [32] 

Judiesch  et  al.  (1992)  make  several  arguments  and  present  a 
considerable  amount  of  literature  supporting  their  assumptions  as 
reasonable.  Several  of  our  reviewers  point  out,  however,  that  they 
do  NOT  expect  pay  for  military  jobs  to  accurately  reflect  the  value 
of  those  jobs.  First,  our  reviewers  tend  to  believe  that,  overall, 
the  pay  in  the  military  is  less  than  that  for  comparable  civilian 
jobs.  They  also  believe  that  those  in  the  more  complex  jobs  are 
more  underpaid  than  those  in  the  less  complex  jobs.  Indeed,  a 
previous  section  of  this  report  supports  their  point  of  view  (see 
section  entitled  "Estimating  Individual  Marginal  Utility  in 
Dollars").  If  we  ignore  these  effects  in  our  study,  then  we  lose 
the  increased  economic  benefit  which  really  does  exist  for 
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classifying  high  ability  personnel  into  the  more  complex  jobs.  In 
other  words,  ignoring  these  effects  provides  a  conservative 
estimate  (i.e.,  an  underestimate)  of  utility.  Therefore,  we 
employed  formulas  [31]  and  [32]  in  our  basic  analyses  without 
modifying  them  to  compensate  for  possible  underpayment  in  military 
jobs.  A  later  study  could  conduct  sensitivity  analyses  and 
incorporate  procedures  to  offset  these  pay  issues  and  assess  the 
impact  of  these  assumptions. 

We  computed  Y  from  a  military  financial  database  following 
formulas  [31]  and  [32].  We  received  a  database  (from  Mr.  Lou  Pales 
of  the  Defense  Manpower  Data  Center,  DMDC,  in  Monterey)  which 
includes  financial  data  on  over  2.4  million  personnel  in  the 
military  during  fiscal  year  1992.  We  also  received  documentation 
from  the  Pentagon  Office  of  Public  Affairs  indicating  that  the 
total  Department  of  Defense  expenditures  (i.e.,  the  total  revenue 
value  in  equation  31)  for  fiscal  year  1992  amounted  to  $282.6 
billion.  Using  the  DMDC  financial  data  base,  we  calculated  the  job 
payroll  for  each  military  job  and  the  total  military  payroll  (just 
over  $50.8  billion).  Entering  these  values  into  equation  31,  we 
computed  the  job  values  for  each  military  job.  To  calculate  Y,  we 
divided  job  values  by  the  number  of  work  years  worked  for  that 
military  job  during  fiscal  year  1992. 

Table  13  shows  the  per  person  job  values  (Y)  for  the  17 
occupational  specialties  in  our  sample.  That  table  also  shows  the 
job  complexity  categories,  the  SDp  values  (one  value  per  complexity 
category) ,  and  the  SDy  values  (job  value  *  SDp) . 
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Insert  Table  14  about  here. 


Developing  Job  Complexity  Categories 

As  we  developed  estimates  of  marginal  utility  in  dollars,  the 
procedures  above  for  estimating  SDy  required  that  we  obtain  job 
complexity  levels  for  our  sample  of  18  jobs  as  well  as  all  entry- 
level  MOSs.  Several  sections  above  refer  to  these  job  complexity 
categories  or  levels.  This  section  describes  how  we  developed 
those  categories. 

DMDC  maintains  data  which  delineates  occupational  specialties 
in  terms  of  ratings  on  the  data-people-things  scale  from  the 
Dictionary  of  Occupational  Titles  (U.s.  Department  of  Labor,  1977). 
Using  a  procedure  defined  by  Hunter  (1980)  ,  Abrahams  et  al.  (1993) 
used  these  data  to  develop  complexity  ratings  for  the  jobs  in  our 
sample.  Complexity  is  defined  as  the  level  of  cognitive 
information  processing  demands  imposed  by  the  job.  Abrahams  et  al. 
determined  that  all  the  18  jobs  fall  into  one  of  four  "complexity" 
levels.  In  a  similar  study,  Schmidt  et  al.  (1987)  used  a  variant 
of  "this  procedure  but  chose  to  combine  two  of  the  complexity 
categories.  They  worked  with  three  complexity  levels  in  their 
study  of  Navy  jobs.  Also,  Hunter  et  al.  (1990)  used  a  similar 
procedure  and  worked  with  three  levels  in  their  analysis  of  studies 
in  the  literature.  We  worked  with  three  complexity  levels  in  this 
study  so  that  we  could  readily  use  published  SDp  values  associated 
with  each  complexity  level  and  more  readily  compare  findings  from 
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this  study  with  findings  of  Schmidt,  Hunter,  and  their  colleagues. 
Because  we  worked  with  three  rather  than  four  levels  of  complexity , 
we  expected  our  basic  analyses  to  yield  more  conservative  estimates 
of  marginal  utility. 

Other  Cost  Considerations 

As  described  above,  the  Ford-Fulkerson  software  employed  the 
PP  scores  as  input  for  each  individual  and  optimized  the  assignment 
of  individuals  to  jobs.  In  this  section,  we  begin  by  describing 
several  other  cost  considerations  which  we  argue  to  ignore  in  this 
study.  Essentially,  ignoring  cost  differences  between  batteries 
due  to  the  testing  programs,  attrition,  recruiting,  and  processing 
efficiency  leads  to  either  no  impact  or  a  conservative  solution. 
This  subsection  concludes  by  supporting  our  initial  contention  to 
use  formula  (4)  to  compute  the  total  marginal  utility  per  job 

(AUTotal  Per  Job)  • 

To  compare  the  utilities  of  competing  batteries,  one  might 
consider  including  values  for  such  costs  as:  the  testing  program 
(c) ,  attrition  (a) ,  recruiting  (r) ,  and  processing  efficiency  (p) . 
For  a  given  job  in  our  sample,  then,  we  would  adjust  equation  (4) 
to  include  these  terms  as  follows: 

4“Tot.l  Per  jcb  -  N.  45.  '  =  '  »  '  +  P  (®> 

For  the  moment,  let's  ignore  the  initial  investment  costs 
(e.g.,  investment  in  computers)  required  to  implement  a  new 
battery.  (In  a  later  section,  we  discuss  including  this  initial 
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investment.)  At  this  point,  let's  examine  the  potential  impact  of 
ignoring  each  of  these  other  cost  considerations  to  determine 
whether  this  would  provide  more  conservative  estimates  of  utility 
for  a  new  battery. 

For  example,  an  important  benefit  of  a  computerized  adaptive 
battery  is  a  reduction  in  Test  Administrator  time.  This  occurs 
because  examinees  complete  those  tests  in  less  time.  However,  our 
comparisons  between  pairs  of  batteries  held  test  time  constant. 
This  practically  eliminated  any  differences  in  c  between  proposed 
batteries.  In  this  case,  we  envisioned  no  bias  in  the  utility 
estimate  by  ignoring  this  term. 

Similarly,  by  ignoring  the  costs  for  attrition  (a),  recruiting 
(r) ,  and  processing  efficiency  (p) ,  we  expected  either  no  bias  or 
a  more  conservative  utility  estimate.  Theoretically,  the  battery 
that  achieves  a  higher  average  level  of  training  or  job  performance 
will  also  produce  less  academic  attrition  during  training  and  less 
attrition  on  the  job.  In  turn,  reduced  attrition  means  reduced 
costs  for  recruiting.  Regarding  processing  efficiency  (p) ,  our 
comparisons  left  no  differences  between  batteries  in  proposed  test 
time.  This  left  little  opportunity  for  differences  in  processing 
efficiency  between  batteries.  It  is  possible,  however,  that  the 
computerization  of  testing  and  scoring  may  offer  opportunities  to 
increase  this  value  for  new  computerized  tests.  Hence,  not 
including  p  in  the  equation  might  have  provided  a  slight  bias 
against  new  computerized  tests.  In  summary,  the  net  effect  of 
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ignoring  c,  a,  r,  and  p  provided  more  conservative  estimates  of 
utility  gains. 

Our  SOW  requires  that  we  develop  a  dollar  metric  that  will 
allow  improvements  in  validity  or  classification  efficiency  "to 
compensate  for  decrements  in  processing  efficiency."  Since  the  SOW 
does  not  require  us  to  develop  estimates  for  these  other  terms,  we 
used  equation  (4)  but  not  (5).  Since  the  SOW  does  require  us  to 
show  how  to  use  the  other  terms  with  our  components  in  future 
calculations  of  utility,  we  developed  equation  (5)  to  provide  this 
information.  However,  note  that  if  a  new  battery  shows  utility 
over  the  existing  one  without  considering  the  cost  savings  for  c, 
a,  r,  and  p,  then  that  utility  estimate  is  probably  a  conservative 

one. 

Generalizing  Prom  This  Sample  of  Jobs 

The  SOW  requires  us  to  compare  the  utility  of  various  pairs  of 
batteries.  Using  equation  (4)  and  ignoring  the  terms  in  equation 
(5)  for  c,  a,  r,  and  p,  we  calculated  the  total  marginal  utility  of 
one  battery  (AUjotaKa))  total  marginal  utility  of  a  second 

(AU^Qtai{b))  battery  separately.  Where  the  results  of  significance 
testing  supported  the  difference  between  the  benefits  of  the  two 
batteries,  we  calculated  marginal  utilities  and  net  present  values. 
In  determining  the  utility  of  one  battery  over  another,  we  needed 
to  generalize  from  our  sample  of  18  jobs  to  all  622  entry-level 

MOSS. 

yiQ  generalized  by  assuming  that  all  ]obs  within  the  same 
complexity  level  have  the  same  SDp  value.  First,  we  categorized 
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all  jobs  by  complexity  level  to  assign  the  appropriate  SDp  value  to 
each.  Then,  we  multiplied  each  job's  Y  value  by  its  SDp  value  to 
obtain  the  SDy  value  for  the  job.  What  remained  was  to  estimate 
either  r^^y  and  or  the  product  of  r^y  and  Z^g  (i.e.,  MPP)  for  all 
622  entry-level  MOSs  for  each  battery.  We  made  these  estimates  by 
using  the  MPP  from  each  complexity  level  in  our  18-school  sample. 

First,  we  describe  how  we  estimated  a  weighted  MPP^  (WMPP^) 
for  a  new  battery.  We  began  by  finding  the  mean  of  the  MPP^  scores 
in  each  complexity  category  in  our  sample  of  18  jobs.  For  example, 
if  complexity  category  one  (Cl)  has  two  jobs  in  our  sample,  then  we 
computed  a  weighted  mean  MPP^^  score  for  that  category  as: 


MPP. 


»{C1) 


(W,„  •MPP,,..)  * 

<"(i)  *"(2)) 


Where  the  subscripts  and  ^(2)  indicate  values  for  a  particular 
battery  (^)  and  for  a  particular  school  or  ^2))*  I”  this  way, 
we  computed  an  MPP  for  each  of  the  three  complexity  categories  for 
each  battery. 

We  used  the  MPP  for  each  complexity  level  from  our  18 -school 
sample  to  generalize  to  the  622  entry-level  MOSs.  Table  15 
provides  an  overview  of  how  we  accomplished  this  generalization  and 
computed  AU^otai  by  battery.  For  each  of  the  622  entry-level  MOSs, 
we  applied  equation  (3)  and  computed  AUj^tai  (Per  school)  ®ach 
battery  as  the  product  of  MPP,  SDy,  and  N.  Recall  that  N  is  the 
expected  quota  for  each  school  for  one  fiscal  year,  MPP  is  the 
estimated  mean  performance  for  all  jobs  in  a  complexity  level,  and 
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SDy  is  the  SDp  value  for  the  job's  complexity  level  times  the  mean 
job  value. 

After  computing  this  value  for  each  job,  we  accumulated  across 
all  622  entry-level  MOSs  for  each  battery  and  computed  the  total 
marginal  utility.  Table  15  shows  these  as  ^^lotaiiia.)  ^^Totai(b) 

for  batteries  (a)  and  (b) .  Hence,  the  total  net  benefit  (TNB)  to 
the  military  for  using  one  battery  instead  of  the  other  is  the 
difference  between  these  two  total  marginal  utilities: 

TNB  =  ^Ujotal(a)  “  ^^Total(b) 


Insert  Table  15  about  here. 


Present  Value  Analysis 

Developing  the  Procedures.  In  an  analysis  of  the  net  present 
value  (NPV) ,  Cascio  (1989)  points  out  that  NPV  must  be  greater  than 
zero  to  provide  economic  justification  for  investing  in  a  personnel 
program.  In  eguation  (8)  below,  we  substituted  our  Total  Net 
Benefit  (TNB)  term  where  Cascio  uses  a  Benefit  (B)  term: 


NPV  =  -C  +  * 


t-i 


(1+i) 


>  0 


(8) 
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where, 

C  is  the  initial  setup  cost  of  the  second  battery  (for 
existing  batteries ,  we  assumed  no  initial  setup  cost) , 
TNB  is  the  total  net  benefits  (in  incremental  cash  flow)  of 
the  new  program  over  the  existing, 
i  is  the  discount  rate  (currently  7%),  and 
n  is  the  number  of  periods  (years,  t)  over  which  the 

benefits  of  the  new  battery  last. 

Notice  that  the  definition  of  C  includes  an  assumption  that 
existing  batteries  require  no  initial  setup  cost.  Several 
comparisons  outlined  in  the  SOW  call  for  modifying  the  first 
battery  in  some  small  way.  Therefore,  ignoring  those  setup  costs 
biased  the  comparisons  in  favor  of  those  first  batteries  and 
results  in  conservative  estimates  of  marginal  utility  for  a  new 
battery. 

For  a  discount  rate  (i) ,  we  used  the  7%  rate  set  forth  as  the 
government  discount  rate  in  0MB  Circular  A-94  (1992)  .  As  Zimmerman 

(1980)  explains,  an  appropriate  government  discount  rate  must 
consider: 

".  .  .  the  value  of  opportunities  which  the  private 

sector  must  pass  by  when  resources  are  withdrawn  from 
that  sector.  A  government  project  is  desirable  if,  and 
only  if,  the  value  of  the  net  benefits  it  promises 
succeeds  the  cost  of  the  lost  productive  opportunities 
which  that  investment  causes.”  (p.  9-i) 

other  words,  the  government  policy  on  discount  rates  encourages 
the  use  of  discounting  beyond  simply  offsetting  the  effects  of 
inflation. 
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A  final  issue  to  consider  regarding  the  variables  in  equation 
(8)  is  the  value  of  n,  the  number  of  periods  over  which  the 
benefits  of  the  new  battery  will  last.  Since  the  ASVAB  was 
initially  implemented  on  January  1,  1976  (ASVAB  Working  Group, 
1980) ,  the  ASVAB  itself  has  enjoyed  nearly  18  years  of  use  in 
various  forms.  During  the  17.75  years  between  January  1976  and 
October  1993,  DoD  had  developed  and  retired  4  major  versions  of 
ASVAB  Forms:  5,  6,  &  7;  8,  9,  &  10;  11,  12,  &  13;  and  15,  16, 

&  17.  (They  scheduled  Forms  20,  21,  &  22  for  implementation  in 
October  1993.)  Four  versions  over  17.75  years  yields  an  average 
life  of  about  4.44  years  per  version.  For  the  purposes  of  this 
study,  we  initially  set  n  =  4  when  computing  NPV  using  equation 

(8). 

This  decision  to  take  the  benefit  of  a  new  battery  across  only 
4  years  represents  another  conservative  decision.  It  is 
conservative  because  we  used  4  instead  of  4.44  years.  Secondly,  it 
is  conservative  because  we  considered  changes  to  an  ASVAB  f or m  as 
a  test  change.  (The  last  major  change  to  ASVAB  happened  when  DoD 
implemented  ASVAB  Forms  8,  9,  and  10  in  1980.)  Either  way,  the 
decision  to  use  a  new  battery  for  only  4  years  (or  for  only  4 
annual  cohorts)  cuts  short  the  period  over  which  we  accumulate  the 
TNB  term. 

Finally,  we  considered  the  typical  tenure  of  those  enlisting. 
Using  loss  data  provided  by  DMDC  for  fiscal  years  1987  through 
1989,  we  computed  the  average  median  tenure  of  enlisted  personnel 
as  46.28  months.  This  indicates  that,  on  average  during  those 
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fiscal  years,  half  of  those  enlisting  left  the  military  before  and 
half  left  after  46  months  and  1  week  of  service. 

Implementing  the  Procedures.  In  summary,  we  used  46.28  months 
as  the  median  tenure  of  enlisted  personnel.  Also,  we  used  4  years 
as  the  typical  life  cycle  of  a  DoD  selection  battery.  Table  16 
presents  the  procedures  for  accumulating  TNB  and  determining  its 
present  value  in  FY92  dollars. 


Insert  Table  16  about  here. 


The  first  column  shows  the  fiscal  years  over  which  a  project 
would  implement  a  new  battery  and  then  accumulate  the  benefits. 
NPRDC  anticipates  the  initial  setup  to  occur  through  1997  and  the 
first  cohort  to  begin  during  1997. 

The  second  column  shows  that  we  placed  all  setup  costs  in  the 
first  year  only.  This  clearly  indicates  the  total  projected 
investment  costs.  Not  applying  discounting  procedures  to  the 
initial  setup  costs  provides  another  conservative  assumption.  It 
assumes  that  all  setup  costs  occur  at  the  beginning  of  the  first 
year.  This  makes  the  computerized  or  comparison  battery  appear  to 
have  relatively  higher  initial  setup  costs.  (They  appear  higher  in 
our  analyses  because  we  did  not  discount  them.) 

The  third  column  uses  the  information  that  each  cohort  has  a 
median  tenure  of  3.86  years.  For  each  cohort  year,  this  column 
provides  the  product  of  this  median  tenure  and  the  TNB  term. 
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The  fourth  column  shows  the  appropriate  discount  factor  for 
each  cohort  year.  Since  services  recruit  each  cohort  throughout 
any  given  year,  the  middle  of  a  year  is  the  typical  starting  point 
for  any  given  cohort.  For  the  first  cohort  (FY97  or  the  third 
project  year),  the  typical  ending  point  (i.e.,  the  point  that  half 
the  enlistees  leave  the  military)  is  nearly  four  years  later  or  the 
middle  of  the  seventh  project  year.  Therefore,  this  column  shows 
the  mid-year  discount  factor  for  the  seventh  year  for  the  first 
cohort.  Similarly,  the  typical  ending  points  for  cohorts  two, 
three,  and  four  are  the  middle  of  project  years  eight,  nine,  and 
ten.  Therefore,  the  third  column  presents  mid-year  discount 
factors  for  years  seven  through  ten.  The  final  column  shows  how  we 
accumulated  the  initial  setup  costs  and  the  discounted  TNB  terms  to 
compute  a  net  present  value  of  one  selection  battery  over  another. 

Developing  Initial  Setup  Costs  by  Battery.  Table  17  shows  the 
various  components  of  the  setup  costs  for  each  battery.  Notice 
that  we  included  no  setup  costs  for  paper-and-pencil  batteries 
(i.e.,  batteries  and  B2  in  testing  conditions  A^,  A3,  A5,  and 
Ag) .  Working  closely  with  NPRDC  managers,  we  reasoned  that  current 
program  funding  would  probably  cover  whatever  minor  changes  or 
other  setup  costs  those  batteries  may  require.  Pencil-and-paper 
batteries  may,  however,  require  additional  funding  for  setup  costs. 
Under  such  conditions,  these  analyses  would  underestimate  setup 
costs  for  paper-and-pencil  batteries  and  underestimate  the  present 
value  of  a  computerized  battery.  This  bias  favoring  paper-and- 
pencil  batteries  provides  another  conservative  assumption. 
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Insert  Table  17  about  here. 


Vicino,  Hetter,  Rafacz,  Segall,  and  Unpingco  (1993)  provided 
the  data  for  the  setup  costs  of  computerized  tests  shown  in  Table 
17.  Most  of  the  costs  appearing  in  Table  17  are  the  costs  for 
fiscal  years  1995  through  1997  shown  in  Vicino  et  al.  (page  8,  the 
table  showing  the  funding  profile  for  Scenario  C:  Desktop/Portable 
Mix) .  We  did  not  include  the  normal  operating  costs  shown  in  that 
table  for  0PM  Test  Administrators  and  the  United  States  Military 
Entrance  Processing  Command  (USMEPCOM) ,  since  these  costs  would 
exist  for  any  of  the  alternatives  and  are  not  setup  costs. 

Foot  note  number  one  to  Table  17  identifies  the  few  costs  that 
vary  directly  with  test  completion  time:  equipment  and  freight, 
maintenance,  and  computer  support/supplies.  The  first  category  has 
the  largest  costs.  These  include  the  costs  for  purchasing  micro¬ 
computers  and  shipping  them  to  the  required  sites.  Vicino  et  al. 
(Appendix  D,  1993)  describe  the  model  developed  (which  uses  a 
completion  time  of  two  hours)  to  describe  machine  requirements. 
This  model  identifies  the  need  to  vary  the  number  of  machines 
with  the  test  completion  times  so  that  the  average 
applicant  waiting  time  remains  less  than  10  minutes.  Working  with 
this  information,  we  interpolated  or  extrapolated  from  data  in 
Vicino  et  al.  to  develop  the  cost  figures  for  our  Table  17.  To 
compute  the  figures  in  Table  17  for  Testing  Condition  Aj^  for 
example,  we  took  the  ratio  of  the  test  time  in  Aj  to  the  test  time 
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in  Vicino  et  al.  (1.66/2.00)  times  the  cost  figures  in  their 
report.  Similarly,  we  used  the  ratio  of  2.25/2.00  for  condition  A4 
and  the  ratio  of  3.00/2.00  for  conditions  A7  and  Ag. 

Attachment  (5)  of  Vicino  et  al.  (1993)  provided  the  cost  of 
Response  Pedestals  shown  in  Testing  Condition  Ag.  Their  table  for 
Scenario  C  —  Desktop/Portable  Mix  showed  a  cost  of  $2.1  million 
for  response  pedestals. 

sensitivity  and  Break-even  Analyses 

Earlier  in  this  plan,  we  describe  how  we  used  statistical 
tests  to  assess  whether  one  battery  yields  improved  predicted 
performance  over  another.  Once  we  established  the  relative 
standing  of  any  two  batteries  on  predicted  performance,  the  utility 
analyses  above  use  a  common  metric  (dollars)  to  guantify  the  extent 
of  improvement.  However,  existing  utility  models  contain  no 
parameters  reflecting  variability  in  the  estimates  of  the  various 
components  of  the  utility  models.  Recognizing  this,  we  originally 
planned  (lack  of  time  prevents  us  now)  to  use  sensitivity  analyses 
and  break-even  analyses  to  assess  the  impact  of  uncertainty  in  our 
utility  analyses. 

In  sensitivity  analyses,  we  would  have  varied  each  of  the 
utility  parameters  from  a  low  value  to  a  high  value  while  holding 
other  parameter  values  constant.  By  plotting  and  examining  the 
resulting  utility  estimates,  we  could  have  identified  which  of  the 
parameters'  variability  has  the  greatest  effect  on  the  total 
utility  estimate. 
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In  break-even  analysis,  we  could  have  calculated  the  lowest 
value  of  any  individual  utility  parameter  (or  combination  of 
parameters)  that  still  yields  a  positive  Total  Net  Benefit  (TNB) 
term.  Any  parameter  value  exceeding  this  "break-even”  value  would 
provide  positive  marginal  utility.  if  sensitivity  analyses  are 
undertaken  in  the  future,  investigators  should  plot  and  examine  the 
resulting  utility  estimates. 

As  Boudreau  (1988)  points  out,  break-even  analysis  allows 
decision  makers  to  determine  the  critical  values  for  utility 
parameters  that  could  change  the  decision  and  determine  whether  to 
pursue  further  refinements  in  measurement.  Where  sensitivity 
analyses  identify  parameter  break-even  points  that  are 
substantially  below  the  values  we  obtained  in  this  study,  then  such 
findings  will  support  the  insensitivity  of  our  findings  to  the 
uncertainty  of  the  parameter  estimates.  Where  parameter  break-even 
points  are  relatively  close  to  the  values  we  obtained,  then  such 
findings  will  be  sensitive  to  the  uncertainty  of  our  parameter 
estimates.  Under  these  latter  conditions,  we  may  want  to  consider 
refinements  in  the  measurements  to  better  evaluate  the  utility  the 
military  would  actually  experience. 
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Results  and  Discussion 


overall  Comparison  of  Eight  candidate  Batteries 

Using  the  regression  equations  developed  for  each  candidate 
battery,  we  conducted  preliminary  analyses  of  their  expected 
effectiveness  prior  to  generating  optimal  school  assignment 
solutions.  In  these  analyses,  we  compared  the  eight  testing 
conditions  on  various  indices  that  either  contribute  to  or  directly 
estimate  their  relative  classification  efficiency.  Table  18 
displays  these  indices  for  the  eight  testing  conditions.  The  first 
two  columns  of  Table  18  list  the  testing  conditions  (TC)  and  their 
corresponding  battery.  The  third  column.  Time,  indicates  the 
number  of  minutes  for  each  battery. 


Insert  Table  18  about  here. 


The  next  colximn  of  Table  18,  Phi,  provides  Horst's  index  of 
differential  prediction  efficiency,  <t>,  for  each  of  the  batteries. 
Although  this  index  has  no  simple  interpretation,  the  larger  the 
value,  the  greater  the  differential  prediction  efficiency  of  the 
battery.  As  Table  18  indicates,  this  index  becomes  larger  as  the 
battery  becomes  longer ,  or  more  complex  ( i . e . ,  includes  more 
tests)  .  Because  the  batteries  B2,  B4  and  B5  are  computer 
administered,  some  examinees  take  more  time  and  some  examinees  take 
less  time,  although  the  expected  or  average  completion  time  must 
remain  as  the  Time  column  indicates. 
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Column  R'  presents  the  average  multiple  correlation  for  each 
battery.  The  following  column,  r',  is  the  average  intercorrelation 
among  the  least  squares  prediction  equations  for  the  battery. 
Examination  of  these  two  columns  offers  some  insight  into  Horst's 
process.  Notice  that,  as  the  computerized  testing  conditions  (A2, 
A4,  Ay,  and  Aq)  increase  in  length  or  complexity,  the  value  R'  goes 
up  while  the  value  r'  goes  down.  This  relationship  also  occurs  for 
the  paper-and-pencil  testing  conditions  (A^,  A3,  A5,  and  Ag) . 
Because  Horst's  process  adjusts  the  test  lengths  to  maximize  R' 
while  minimizing  r',  we  expect  this  relationship.  There  are  no 
beta  weights  that  will  yield  a  higher  R'  for  these  batteries  at 
their  final  lengths.  (At  the  completion  of  Horst's  procedure,  the 
beta  weights  for  multiple  absolute  prediction  are  identical  to 
those  for  multiple  differential  prediction) . 

The  AA  column  represents  the  allocation  average  (mean 
score) ,  which  we  estimated  using  Brogden's  (1959) 
procedure.  This  procedure  assumes  equal  quotas  for  each  school. 
We  estimated  these  averages  using  R',  r',  the  selection  ratio,  and 
the  number  of  schools.  The  C  column  presents  the  multiplier  for 
Brogden's  allocation  average  measure.  We  define  the  value  C  as: 

c^R'yfrrp  (9) 

Brogden's  (1959)  equation  assumes  that  the  validities  (R')  are 
the  same  for  all  is  regression  equations,  as  are  their 
intercorrelations  (r').  Nevertheless,  If  we  substitute  the  mean 
values,  R'  and  r',  in  equation  (1),  it  provides  a  good 
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approximation  for  Brogden's  (1959)  measure.  Notice  that,  as 
expected,  C  increases  as  the  battery  increases  in  length  and 
complexity.  The  larger  this  index  C,  the  greater  the  differential 
effectiveness  of  the  battery,  regardless  of  the  selection  ratio. 
This  index  clearly  shows  that  decreasing  intercorrelations  between 
composites  provides  real  gains  in  Classification  efficiency  even  if 
absolute  validity  remains  constant. 

As  indicated  earlier,  the  AA  column  provides  Brogden's  (1959) 
Allocation  Average.  These  are  the  average  expected  criterion  z- 
scores  for  selectees.  To  obtain  these  AA  values,  we  first 
calculated  Brogden's  AA  index  assuming:  1)  18  schools,  2)  a 
selection  ratio  of  .6523913,  3)  a  composite  validity  of  1.00,  and 
4)  a  composite  intercorrelation  of  .00.  Then  we  multiplied  this 
common  AA  by  each  testing  condition's  unigue  C  value  to  obtain  the 
Brogden's  AA  values  in  Table  18.  As  this  table  illustrates,  this 
average,  which  is  the  average  expected  criterion  z-score  for 
selectees,  increases  as  the  battery  increases  in  length  and 
complexity.  We  would  expect  about  1/5  of  a  standard  deviation 
increase  in  performance  among  selectees  using  the  best  battery,  Ag, 
as  compared  to  the  worst  battery,  A^. 

It  is  worth  noting  that  changes  in  intercorrelations  among  the 
prediction  equations  appear  to  have  a  greater  impact  on  the 
allocation  average  than  changes  in  the  absolute  validity  of  the 
equations.  That  is,  from  treatment  A^  to  treatment  Ag^  the  average 
multiple  R  (i.e.,  R'  in  Table  18)  goes  from  .6908  to  .7449.  This 
is  a  change  of  .0541,  or  a  gain  of  just  under  8%.  However,  the 
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allocation  average  goes  from  .3912  to  .6285,  which  is  a  change  of 
.2337,  or  a  gain  of  just  under  60%.  This  would  suggest  that 
concentrating  on  absolute  (or  incremental)  validity  alone  in 
assessing  or  selecting  a  battery  of  tests  may  fail  to  realize  the 
gains  in  classification  efficiency  that  might  be  possible. 

Column  G  presents  the  expected  allocation  average  if  we 
assigned  all  individuals  using  a  single  factor,  such  as  G,  and  if 
the  average  R  for  this  factor  were  the  value  given  in  the  R' 
column.  Table  18  shows  that  differential  effectiveness  only 
slightly  increases  the  allocation  average  of  the  worst  battery,  A^. 
However,  battery  Aq  shows  a  considerable  gain  in  differential 
effectiveness. 

Recall  that,  for  each  of  the  eight  testing  conditions,  we 
conducted  eight  replications,  resulting  in  64  unique  assignment 
solutions.  For  each  of  these  64  solutions,  we  computed  a  mean 
predicted  performance  value  (see  Appendix  H) .  Table  19  presents 
the  analysis  of  variance  of  these  64  MPPs.  We  analyzed  these  data 
using  a  mixed  model,  with  blocks  random  and  treatments  fixed.  For 
this  model,  the  appropriate  error  term  for  testing  the  significance 
of  differences  between  testing  conditions  is  the  mean  square  (MS) 
for  treatments  X  blocks  (Edwards,  1972,  p.  240). 


Insert  Table  19  about  here. 


As  Table  19  indicates,  the  testing  condition  factor  is 
statistically  significant,  £(7,  49)  =  50.46.  We  may  use  eta 
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squared  to  estimate  the  proportion  of  total  variance  accounted  for 
by  the  testing  condition  factor  (McNemar,  1962,  p.  279).  This 
statistic  is  descriptive  rather  than  inferential  in  this  instance. 
Eta  squared,  computed  by  SSA/SST,  equals  .4978.  Thus,  the  testing 
condition  factor  accounts  for  approximately  half  of  the  total 
variance  in  predicted  performance  values.  The  mean  predicted 
performance  values  for  the  eight  batteries  range  from  .60  to  .74, 
reflecting  a  23%  increase  in  expected  performance. 

Specific  Contrasts  of  Candidate  Batteries 

Because  the  overall  £-test  revealed  significant  differences 
between  testing  conditions,  we  conducted  further  significance 
tests.  These  tests  involve  the  pairs  of  batteries  hypothesized  to 
differ  in  classification  efficiency.  Table  20  shows  the 
significance  test  results  for  the  six  planned  comparisons.  These 
comparisons  test  the  directional  hypotheses  we  made  before 
examining  any  results.  Consequently,  the  values  in  Table  20  do  not 
require  corrections  in  alpha  level  for  capitalization  on  chance. 
We  used  one-tail  t  tests  to  assess  significance  (see  McNemar,  1962, 
p.  285). 


Insert  Table  20  about  here. 


The  first  comparison  (SOW  4.4.1)  of  candidate  test  batteries 
contrasts  the  classification  efficiency  of  a  P&P-ASVAB  to  that  of 
a  CAT-ASVAB.  One  hundred  minutes  of  testing  time,  optimally 
allocated  to  the  individual  tests,  are  available  for  each  battery. 
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We  hypothesized  that  the  computerized  battery  would  out-perform  the 
P&P  battery  because  its  greater  reliability  will  result  in  greater 
validity.  As  Table  21  indicates,  the  CAT-ASVAB  provided  greater 
classification  efficiency  than  the  P&P-ASVAB.  The  resulting 
allocation  averages  of  .65  for  CAT-ASVAB  and  .60  for  P&P-ASVAB  are 
statistically  significant,  t(49)  =  4.96,  p  <  .001.  We  may 
attribute  this  gain  in  classification  efficiency  to  the 
computerized  mode  of  administration. 


Insert  Table  21  about  here. 


The  second  guestion  (SOW  4.4.2)  examines  the  difference 
between  an  expanded  P&P-ASVAB  battery  and  an  expanded  CAT-ASVAB 
battery.  Specifically,  we  compared  a  P&P  battery,  containing  ASVAB 
plus  three  P&P-format  ECAT  tests,  to  a  computerized  battery, 
containing  CAT-ASVAB  and  six  non-pedestal  ECAT  tests.  In  this 
comparison,  we  optimally  allocated  135  minutes  to  the  tests  in  each 
battery.  The  computerized  battery  contains  all  the  tests  of  the 
P&P  battery  plus  three  additional  tests,  which  could  only  increase 
the  utility  of  the  computerized  battery.  Therefore,  we 
hypothesized  that  the  expanded  P&P-ASVAB  would  have  lower  utility 
than  the  expanded  CAT-ASVAB.  As  expected,  the  expanded 
computerized  battery  out-performed  the  expanded  P&P  battery,  t(49) 
“  7.11,  p  <  .001.  As  Table  21  shows,  the  allocation  average  of  the 
P&P  battery  is  .62  and  that  of  the  computerized  battery  is  .69. 
This  gain  may  be  due  to  both  mode  of  administration  and  the 
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additional  three  tests,  two  of  which  measure  non-verbal  reasoning 

and  one  measuring  spatial  ability. 

The  third  comparison  (SOW  4.4.3)  contrasts  the  classification 
efficiency  of  a  P&P-ASVAB  and  that  of  an  expanded  P&P-ASVAB. 
Specifically,  the  expanded  ASVAB  includes  three  ECAT  tests  in  P&P 
format.  For  both  batteries,  180  minutes  are  optimally  allocated  to 
the  individual  tests.  For  this  contrast,  we  hypothesized  that  the 
expanded  ASVAB  (i.e.,  ASVAB  plus  three  ECAT  tests)  would  provide 
greater  classification  efficiency  than  would  the  ASVAB  alone.  The 
addition  of  individual  tests,  with  test  time  set  to  maximize  their 
contribution,  can  only  increase  efficiency.  As  Table  21  reveals, 
the  expanded  ASVAB  yielded  a  higher  classification  efficiency  than 
the  ASVAB  alone.  The  resulting  allocation  averages  are  .62  and  .64 
for  ASVAB  and  the  expanded  ASVAB,  respectively.  Although  the 
difference  (.02)  between  these  averages  is  the  smallest  observed 
among  the  six  planned  comparisons,  it  is  statistically  significant, 
t(49)  =  1.92,  E  <  .05.  This  increase  in  classification  efficiency 
results  from  the  additional  three  tests,  which  primarily  measure 
spatial  ability. 

The  fourth  contrast  (SOW  4.4.4)  of  test  batteries  compares  a 
P&P  battery,  containing  a  P&P-ASVAB  plus  three  ECAT  P&P-format 
tests,  to  a  computerized  battery,  containing  a  CAT-ASVAB  and  six 
non-pedestal  ECAT  tests.  This  comparison  involves  batteries  which 
are  also  in  our  second  hypothesis  (4.4.2).  However,  for  this 
fourth  comparison,  we  optimally  allocated  180  minutes  to  the 
individual  tests.  Because  the  computerized  battery  includes  all 
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the  tests  of  the  P&P  battery  plus  three  additional  tests,  as  with 
our  second  hypothesis,  we  expected  the  computerized  battery  to 
provide  greater  classification  efficiency.  As  Table  21  shows,  the 
computerized  battery  out-performed  the  P&P  battery.  The  allocation 
averages  of  ,71  and  .64  for  the  two  batteries  (respectively)  are 
significantly  different,  t(49)  =  7.29,  p  <  .001.  Like  the  results 
from  our  second  hypothesis,  this  gain  may  be  due  to  both  mode  of 
administration  and  the  additional  three  tests,  two  of  which  measure 
non-verbal  reasoning  and  one  measuring  spatial  ability. 

The  fifth  comparison  (SOW  4.4.5)  contrasts  two  computerized 
batteries:  a  CAT-ASVAB  plus  the  six  non-pedestal  ECAT  tests,  and 
a  CAT-ASVAB  plus  a  full  ECAT.  We  optimally  distributed  180  minutes 
to  the  individual  tests  in  each  battery.  Because  it  has  three 
additional  tests,  we  hypothesized  that  the  battery  containing  the 
CAT-ASVAB  plus  a  full  ECAT  would  have  the  higher  classification 
efficiency.  As  expected,  battery  containing  CAT-ASVAB  plus  the 
full  ECAT  revealed  significantly  higher  classification  efficiency 
when  compared  to  the  CAT-ASVAB  plus  non-pedestal  ECAT  battery, 
Jt(49)  =  3.20,  E  <  .01.  Table  21  displays  allocation  averages  of 
.74  and  .71  for  these  batteries,  respectively.  This  gain  in 
classification  efficiency  may  be  the  result  of  the  unique  variance 
measured  by  the  three  pedestal  tests,  two  of  which  involve 
psychomotor  skills  and  one  measuring  perceptual  speed. 

Finally,  the  sixth  question  (4.4.6)  examines  the  difference 
between  the  utility  of  a  P&P  battery,  containing  a  P&P-ASVAB  plus 
three  P&P-format  ECAT  tests,  and  a  computerized  battery,  containing 
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a  CAT-ASVAB  plus  all  ECAT  tests.  For  both  batteries,  we  optimally 
distributed  180  minutes  to  the  individual  tests.  Here,  the 
computerized  battery  contains  all  the  tests  of  the  P&P  battery, 
plus  six  additional  tests.  Consequently,  we  hypothesized  that  the 
P&P  battery  would  have  lower  utility  than  the  computerized 
battery.  As  Table  21  shows,  the  computerized  battery  out-performed 
the  P&P  battery.  The  allocation  averages  of  .64  and  .74  for  the 
P&P  and  the  computerized  battery  (respectively)  are  significantly 
different,  t(49)  =  10.49,  p  <  .001.  This  gain  in  classification 
efficiency  is  due  to  the  computerized  administration  mode,  and  the 
additional  six  tests.  Of  these  six  tests,  two  measure  non-verbal 
reasoning,  one  involves  spatial  ability,  two  assess  psychomotor 
skills,  and  one  measures  perceptual  speed. 

The  results  of  these  contrasts,  and  an  inspection  of  mean 
predicted  performance  values  in  Table  21,  indicate  that  changes  in 
battery  length,  composition,  and  mode  of  administration  lead  to 
statistically  and  practically  significant  increases  in 
classification  efficiency.  These  increases  translate  into 
practical  gains  in  expected  performance.  Such  gains  result  from 
the  combined  effects  of  increases  in  validity  and  decreases  in 
intercorrelation  of  the  predicted  criterion  scores  of  each  testing 
condition. 

The  maximum  possible  gain  in  MPP  between  any  two  of  these 
candidate  test  batteries  is  the  .14  difference  observed  between  a 
100-minute  P&P  ASVAB  battery  and  a  180-minute  CAT-ASVAB  plus  full 
ECAT  battery.  These  two  batteries  differ  in  terms  of  time, 
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composition,  and  mode  of  administration.  Because  these  three 
factors  (time,  composition,  mode  of  administration)  are  confounded, 
it  is  not  possible  to  precisely  determine  each  factor's 
contribution  to  the  .14  increase  in  mean  predicted  performance. 
However,  inspection  of  the  means  in  Table  21  suggests  that  the  time 
increase  results  in  .02  MPP  points,  computer  administration  results 
in  .05  points,  and  battery  composition  results  in  .07  points. 
Results  of  Utility  Analyses 

MPP  Values  by  Testing  Condition  and  Hypothesis 
Table  22  presents  mean  predicted  performance  (MPP)  values  by 
testing  condition  and  hypothesis.  The  Method  section  on 
Generalizing  From  This  Sample  of  Jobs  describes  how  we  obtained  the 
MPPs  displayed  in  Table  22.  For  every  battery,  we  weighted  each  of 
the  three  MPP  values  obtained  from  our  sample  of  18  jobs  by  the 
number  of  people  in  the  corresponding  complexity  level  for  the  622 
MOSs.  The  MPP  values  shown  in  Table  22  differ  from  previous  values 
in  that  they  include  more  appropriate  weights  for  all  three 
complexity  levels. 

The  622  entry-level  MOSs  provide  greater  opportunity  for 
increases  in  classification  efficiency  than  possible  with  18  jobs. 
To  adjust  for  the  increased  classification  efficiency  expected  with 
increased  assignment  possibilities,  we  multiplied  the  Brogden 
factors  (presented  in  Table  23)  by  the  mean  predicted  performance 
values  in  Table  22.  The  parenthetical  values  in  Table  22  display 
the  Brogden  adjusted  values.  (To  estimate  classification 
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efficiency  and  utility  for  100  jobs,  multiply  the  non-parenthetical 
values  in  Table  22  by  1.29.) 


Insert  Table  22  about  here. 


Table  22  also  displays  specific  comparisons  between  batteries 
in  accordance  with  the  SOW.  Each  row  of  Table  22  represents  a 
specific  study  hypothesis.  The  first  entry  in  each  row  represents 
the  mean  MPP  for  the  first  battery,  while  the  second  entry  in  each 
row  represents  the  second  battery.  The  last  entry  in  each  row  is 
the  "DeltaMpp".  The  "DeltaMpp"  column  represents  the  difference  in 
utility  between  the  two  batteries  for  each  comparison.  The 
positive  values  in  this  column  indicate  that  all  differences  in  MPP 
values  were  in  the  predicted  direction.  Since  the  t-test  performed 
on  the  planned  comparisons  indicate  that  all  differences  were 
significant,  we  applied  the  utility  analysis  procedures  to  all  the 
comparisons  identified  by  the  hypotheses. 


Insert  Table  23  about  here. 


Total  Utilities  by  Testing  Condition  and  Hypothesis 
Table  24  presents  the  total  utilities  (i.e. ,  across  all 
selectees  and  all  schools)  by  testing  condition  and  research 
question.  Tabled  values  are  in  millions  of  fiscal  year  1992 
dollars.  Except  for  the  last  coliunn,  each  column  in  Table  24 
presents  a  AU^otai  value  from  equation  (7).  For  all  rows,  AU^otaKb) 
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is  the  second.  As 


is  the  first  entry  in  the  row  and  AUjotai(a) 
equation  (7)  provides,  the  Total  Net  Benefit  (TNB)  term  (the  final 
entry  in  the  row)  is  AU^otaKa)  minus  AUTotai(b)* 


Insert  Table  24  about  here. 


As  with  Table  22,  the  parenthetical  values  in  Table  24  display 
the  Brogden  adjusted  values.  Only  these  values  in  parentheses 
reflect  the  total  utilities  that  consider  the  classification 
efficiency  achieved  by  having  622  (instead  of  18)  jobs  in  which  to 
classify  selectees. 

Present  Value  Analyses  by  Hypothesis 

Tables  25  and  26  display  the  results  of  present  value 
analyses.  Table  25  presents  analyses  based  on  the  MPP  values 
provided  by  our  sample;  and  Table  26  presents  analyses  based  on  MPP 
values  adjusted  by  the  Brogden  factors.  As  before,  only  the 
Brogden-ad justed  utilities  reflect  the  total  utility  considering 
the  622  entry-level  jobs  in  the  military  as  compared  with  the  18 
jobs  in  our  sample. 


Insert  Table  25  about  here. 


Near  the  top  of  Tables  25  and  26,  the  first  row  containing 
values  presents  the  utility  of  "3.86  years  of  TNB  per  cohort". 
This  is  the  undiscounted  value  of  the  TNB  term  from  Table  24  times 
the  median  tenure  of  enlistees  (about  3.86  years).  Typically,  half 
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the  enlistees  leave  the  military  before  and  half  leave  after 
serving  3.86  years.  Presumably,  many  of  those  staying  in  the 
military  beyond  3.86  years  will  have  gained  more  experience,  earned 
greater  rank,  and  provided  greater  productivity.  Because  all  of 
this  occurs  later  in  time,  however,  their  utilities  should  be 
discounted  more.  Similarly,  those  leaving  prior  to  3.86  years  will 
have  gained  less  experience,  earned  less  rank,  and  provided  less 
productivity.  Because  this  occurs  sooner  in  time,  their  utilities 
should  be  discounted  less.  By  using  the  median  or  "typical”  value, 
we  designed  these  analyses  to  estimate  the  TNB  for  the  entire 
cohort . 


Insert  Table  26  about  here. 


The  rest  of  the  table  entries  show  the  values  for  various 
features  of  the  present  value  analyses  for  each  hypothesized 
comparison.  The  entries  for  fiscal  year  1995  show  the  initial 
setup  costs.  As  mentioned  in  the  Method  section  of  this  report,  we 
did  not  discount  these  values.  This  has  the  effect  of  treating 
these  initial  investment  costs  as  occurring  all  at  the  beginning  of 
1995.  Because  DoD  would  likely  spread  these  costs  over  several 
years,  this  procedure  over-estimates  these  investment  costs  and 
under-estimates  the  utility  of  the  batteries  hypothesized  as  more 
effective . 

Costs  presented  in  the  tables  for  fiscal  years  1997  through 
2000  show  the  discounted  (or  present)  values  of  the  3.86  years  of 
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TNB  per  cohort.  These  figures  are  the  product  of  the  discount 
factors  from  Table  23  times  the  3.86  years  of  TNB  per  cohort  shown 
in  these  tables.  The  final  row  sums  across  fiscal  years  1995  to 
2000  to  obtain  the  net  present  values  by  hypothesized  comparisons. 
These  show  the  present  values  (net  of  investment  costs)  of  the 
total  utility  of  the  hypothesized  battery  for  each  comparison. 

As  stated  above,  results  of  statistical  analyses  showed  that 
the  hypothesized  batteries  provided  significantly  higher  MPPs  for 
each  of  the  six  comparisons.  Since  our  dollar  utilities  are  a 
linear  conversion  of  those  results.  Tables  25  and  26  show  utility 
gains  proportionate  to  the  observed  differences  in  MPP  scores  for 
each  comparison.  When  we  inspect  Table  26,  we  see  that  the  three 
comparisons  showing  the  greatest  utility  gains  are  those  comparing 
CAT  ASVAB  plus  some  form  of  ECAT  against  any  P&P  test.  The 
utilities  range  between  9.6  and  11.6  billion  dollars  for  a  CATr 
ASVAB  plus  some  form  of  ECAT.  The  lowest  utilities  compare  two  P&P 
two  computerized  tests.  The  utilities  of  these  comparisons  ares 
(a)  3.2  billion  dollars  for  a  CAT-ASVAB  and  full  ECAT  over  a  CAT- 
ASVAB  plus  a  non-pedestal  ECAT,  and  (b)  2.3  billion  dollars  for  a 
P&P-ASVAB  plus  P&P-ECAT  over  P&P-ASVAB  alone. 

A  Framework  for  Understanding  utility  Gains 

Utility  gains  from  implementing  new  selection  procedures  in  a 
large  organization  often  appear  incredible  to  managers  responsible 
for  using  those  gains.  By  comparing  the  identified  utility  gains 
with  other  available  standards,  DoD  managers  will  develop  a  better 
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perspective  regarding  the  relative  value  of  those  gains  and  how 
best  to  realize  them. 

This  study's  largest  utility  gain  occurred  with  the  comparison 
between  the  two  180-minute  batteries  of  comparison  six:  the  P&P- 
ASVAB  plus  P&P-ECAT  (battery  A6)  and  the  CAT-ASVAB  plus  full  ECAT 
(battery  A8)  ,  The  subsections  below  help  place  in  perspective  the 
nearly  $11.6  billion  utility  gain  estimated  by  that  comparison. 
The  same  rationale  described  in  those  subsections  applies  to  the 
utility  gain  provided  by  any  of  the  other  six  planned  comparisons 
of  this  study. 

Utility  Gains  as  a  Percent  of  the  DoD  Budget.  One  standard 
for  interpreting  the  value  of  utility  gains  is  to  compare  the 
estimated  $11.6  billion  against  the  annual  DoD  Budget.  For  fiscal 
year  1992,  DoD's  annual  expenditures  amounted  to  $282.6  billion. 
From  this  perspective,  the  entire  $11.6  billion  gain  due  to 
improved  selection  and  classification  provided  by  comparison  six 
equals  only  about  4.1%  (11.6/282.6)  of  one  year's  total  DoD  budget. 

Utility  Gains  Per  Year  as  a  Percent  cf  the  DoD  Budget. 
Continuing  with  this  example,  DoD  would  realize  its  gain  over 
several  cohorts  and  several  years.  Considering  that  the  fourth 
cohort  begins  three  years  after  the  first  and  assuming  that  20 
years  is  the  longest  career  of  any  single  individual,  then  it  takes 
about  23  years  to  accumulate  all  the  utility  gains.  Yes,  the  $11.6 
billion  gain  occurs  over  23  years  and  ec[uals  4.1%  of  the  annual  DoD 
budget.  The  typical  gain  in  any  given  year,  though,  is  less  than 
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two  tenths  of  one  percent  (i.e.,  the  fraction  .00178  or  0.178%)  of 
the  DoD  annual  budget. 

Utility  Gains  as  a  Uniform  Annual  Cash  Flow.  Another  method 
for  interpreting  the  value  of  utility  gains  is  to  consider  the 
average  annual  increment  in  cash  flow  provided  by  the  improved 
procedures.  Considering  the  $11.6  billion  over  23  years,  the 
present  value  of  the  average  annual  increment  in  cash  flow  amounts 
to  $504  million. 

Comparing  These  Utility  Gains  with  Those  of  a  Previous  Study. 
Cascio  and  Ramos  (1986)  report  on  the  utility  gains  of  an  improved 
selection  procedure  for  first-level  managers  in  a  division  of  a 
Bell  operating  company.  They  identified  a  utility  gain  of  $2,676 
per  selectee  per  year.  If  we  divide  the  $504  million  per  year  by 
the  roughly  233,000  military  selectees  per  year,  our  study  shows  an 
average  annual  utility  gain  of  about  $2,163  per  military  selectee. 

Utility  Gains  Expressed  in  salary  Terms  Only.  The  Office  of 
Personnel  Management  (0PM)  has  invested  considerable  time  and 
effort  in  understanding  compensation  strategies  available  to  the 
Federal  Government.  In  the  process,  Schay  (1993)  observes  that 
most  managers  in  the  Federal  Government  are  not  fully  aware  of  the 
cost  of  doing  business.  She  suggests  that  managers  in  the  Federal 
Government  recognize  salaries  as  a  cost  of  doing  business,  but  they, 
do  not  recognize  the  other  direct  and  indirect  business  costs. 

Schay 's  (1993)  observations  suggest  that  DoD  managers  may  have 
difficulty  in  understanding  or  believing  the  military  job  values  we 
computed  (see  Table  11)  .  For  these  managers,  we  suggest  presenting 
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utility  gains  adjusted  to  reflect  salaries  alone.  Since  total  DoD 
salaries  in  fiscal  year  1992  equaled  about  $50.8  billion,  salaries 
equal  about  18%  (50.8/282.8)  of  the  total  budget;  and  the  $11.6 
billion  in  utility  gains  amounts  to  $2.1  billion  in  salaries  alone. 

Realizing  Utility  Gains.  The  improved  selection  and 
classification  of  personnel  into  the  military  can  provide  the 
identified  utility  gains  only  if  DoD  managers  take  action  to 
realize  them.  When  a  new  selection  and  classification  system 
begins  supplying  higher  quality  entry-level  military  personnel, 
these  personnel  will  learn  their  jobs  more  quickly  and  perform  them 
more  efficiently.  If  they  fill  jobs  designed  for  personnel  of 
lesser  ability,  they  may  complete  the  work  for  those  jobs  in  less 
time  and  have  little  to  do  thereafter.  Under  these  conditions, 
however,  they  may  provide  no  real  utility  gains.  In  order  for  DoD 
to  realize  the  utility  gains,  they  must  have  an  opportunity  to 
perform  either  more  of  the  same  level  of  work  or  work  of  greater 
complexity  suited  to  their  greater  ability.  DoD  may  need  to  re¬ 
design  jobs  in  order  to  realize  the  potential  utility  gains. 

For  illustration  purposes,  think  about  realizing  the  potential 
utility  gains  from  improved  selection  and  classification  systems 
through  reduced  hiring.  (This  approach  may  have  some  appeal  under 
the  current  climate  of  downsizing.  This  illustration,  however, 
will  set  the  stage  for  another  means  for  realizing  utility  gains.) 
For  example,  dividing  the  annual  DoD  expenditures  (about  $282.6 
billion)  by  the  number  of  uniformed  military  personnel  (about  2.4 
million)  provides  a  typical  job  value  of  about  $118,000.  One  way 
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to  ensure  realizing  a  projected  $504  million  annually  is  to  reduce 
the  number  of  military  jobs.  In  other  words,  the  permanent 
elimination  of  4,271  military  jobs  ($504  million  divided  by 
$118,000  per  job)  would  ensure  capturing  the  average  annual 
increment  in  cash  flow  projected  from  using  the  new  system. 

Such  an  approach  amounts  to  eliminating  the  jobs  associated 
with  the  0.178%  per  year  utility  gain.  At  this  time,  however,  the 
military  has  already  incurred  a  substantial  amount  of  downsizing 
and  is  already  engaging  in  a  considerable  amount  of  re-designing 
jobs  and  re-structuring  work  flow.  Under  these  circtimstances,  an 
improved  selection  and  classification  system  can  provide  improved 
personnel  resources  for  those  expanded  or  enriched  jobs.  This  will 
enable  the  down-sized  military  forces  to  retain  more  of  their 
original  effectiveness  without  increasing  expenditures. 
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conclusions 


The  results  of  the  present  study  support  the  following 

conclusions: 

1.  Use  of  a  computerized  mode  of  administration  and/or  computer- 
adaptive  tests  increases  classification  efficiency  and  the 
concomitant  classification  utility  over  that  of  a  paper-and- 
pencil  mode  of  administration.  The  increase  in  classification 
efficiency  translates  into  practical  gains  with  respect  to 
expected  performance. 

2.  Inclusion  of  additional  tests  increase  classification 
efficiency  and,  in  turn,  increase  classification  utility,  over 
that  of  the  basic  ASVAB  or  CAT-ASVAB. 

3.  Although  we  did  not  specifically  test  the  significance  of 
manipulating  test  time  among  the  candidate  batteries,  test 
time  appears  to  increase  classification  efficiency  and 
utility.  However,  this  increase  is  small,  relative  to 
increases  due  to  test  computerization  and  expansion. 

4.  We  did  not  compare  an  ASVAB  (or  CAT-ASVAB)  alone  to  an  ASVAB 
(or  CAT-ASVAB)  plus  full  ECAT  battery,  holding  test  time 
constant.  Based  on  the  comparisons  we  do  make,  it  appears 
that  the  addition  of  ECAT  to  an  ASVAB  (or  CAT-ASVAB)  battery 
provides  a  substantial  gain  in  classification  efficiency. 
Since  the  ECAT  demonstrated  little  improvement  in  absolute 
validity,  the  gain  in  classification  efficiency  resulted 
primarily  from  decreases  in  composite  intercorrelations. 


75 


5.  Under  the  assignment  conditions  used  in  this  study,  using  the 
CAT-ASVAB  plus  full  ECAT  instead  of  using  P&P-ASVAB  and  three 
ECAT  tests  provided  the  largest  gain  in  utility.  In  this 
comparison,  we  estimated  the  dollar  value  of  the  utility  gain 
at  about  $11.6  billion  or  about  4.1%  of  the  DoD  expenditures 
for  fiscal  year  1992  (FY  92)  .  Since  DoD  would  accumulate 
these  gains  over  several  years,  the  present  value  of  the 
average  increment  in  cash  flow  would  amount  to  about  $504 
million  annually.  This  amounts  to  0.178%  of  DoD  expenditures 
for  FY  92. 
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Recommendations 


We  recommend  the  development  of  procedures  to  construct  test 
batteries  and  regression  equations  that  will  simultaneously 
maximize  classification  efficiency  while  minimizing  adverse  impact. 
The  optimization  procedures  developed  in  this  study  emphasize 
classification  efficiency  alone.  It  may  be  possible  to  develop  a 
combined  objective  function  that  will  result  in  test  batteries  and 
regression  composites  that  maximize  differential  validity  and 
minimize  adverse  impact. 

We  also  recommend  the  evaluation  of  the  incremental 
classification  efficiency  of  the  existing  ECAT  battery  over  the 
existing  ASVAB  (or  CAT-ASVAB) .  Such  an  evaluation  could  lead  to 
improvements  in  personnel  classification  without  spending 
additional  time  or  funding  on  test  development. 

In  order  to  more  completely  reflect  the  utility  of  increased 
classification  efficiency  achieved  from  improved  predictor 
batteries,  we  recommend  the  development  of  procedures  to  include 
the  non-dollar  job  values  that  the  military  places  on  job 
performance.  At  the  beginning  of  this  study,  we  developed  and 
applied  procedures  to  assign  a  dollar-based  job  value  to  military 
jobs.  During  the  course  of  applying  those  procedures,  we 
discovered  that  military  pay  practices  do  not  completely  reflect 
the  value  the  military  places  on  its  jobs.  By  using  those  dollar- 
based  job  values  in  this  study,  we  computed  dollar-based  utilities 
that  probably  underestimate  the  value  of  job  performance  to  the 
military. 
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During  the  course  of  developing  and  applying  procedures  for 
conducting  utility  analyses,  we  made  literally  dozens  of  decisions 
that  affected  the  results.  In  every  case,  we  chose  the  working 
assumption  that  minimized  the  incremental  utility  of  the  battery 
hypothesized  as  more  effective.  While  such  assumptions  do  produce 
more  conservative  results,  some  may  argue  that  the  many 
conservative  assumptions  produce  results  that  simply  are  too 
conservative.  Therefore,  we  recommend  that  the  Government  conduct 
sensitivity  analyses  to  investigate  the  cumulative  impact  of  these 
conservative  assumptions . 
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Appendix  A 

ASVAB  and  ECAT  Intercorrelations  and  Validities: 
Corrected  and  Uncorrected 
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Table  A-1 


Population  #1  Matrix 

ASVAB  Intercorrelations,  Means  and  Standard  Deviations 
from  the  1991  DOD  Population  of  Applicants 


AR:  WK: 

PC: 

NO: 

CS: 

AS: 

MK: 

MC: 

El: 

MEAN: 

SD: 

GS: 

0.611  0.720 

0.608 

0.275 

0.249 

0.520 

0.554 

0.638 

0.625 

50.615 

8.773 

AR: 

0.596 

0.574 

0.470 

0.395 

0.400 

0.707 

0.613 

0.487 

50.664 

8.645 

WK: 

0.732 

0.324 

0.328 

0.437 

0.497 

0.547 

0.534 

51.311 

7.354 

PCs 

0.396 

0.386 

0.339 

0.500 

0.485 

0.444 

51.156 

7.964 

NO: 

0.640 

0.047 

0.496 

0.228 

0.145 

52.512 

8.013 

CS: 

0.058 

0.408 

0.221 

0.147 

52.266 

7.812 

AS: 

0.197 

0.618 

0.669 

51.409 

9.168 

MK: 

0.494 

0.370 

51.210 

8.689 

MC: 

0.630 

51.941 

9.127 

El: 

50.333 

8.856 
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Table  A-2 


DATAFILE:  JS01.DAT  . 

AC  SCHOOL  STATISTICS 

INTERCORRELATIONS,  MEANS  AND  STANDARD  DEVIATIONS 
FOR  20  VARIABLES  IN  A  SAMPLE  OF  72  OBSERVATIONS. 

2345  6789  10 


1  0.442  0.461  0.311-0.002-0.095  0.526  0.247  0.534  0.5-4 

2  0  315  0.176  0.047  0.033  0.-290  0.402  0.422  0.381 

3  0.442-0.196-0.096  0.271  0.308  0.343  0.410 

4  0.156  0.105  0.402  0.315  0.268  0.248 

5  0.400  0.169-0.020-0.033  0.022 

g  0.144-0.060  0.105-0.065 

7  0.142  0.642  0.637 

o  0.081  0.179 


10 


11  12  13  14 


15  16  17  18  19  20  MEAN: 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 


0.247  0.249 
0.312  0.286 
0.138  0.197 
0.170  0.291 
0.066  0.105 
0.177  0.106 
0.141  0.458 
0.079  0.125 
0.276  0.441 
0.132  0.288 
0.381 


0.331  0.163 
0.411  0.253 
0.242  0.189 
0.206  0.243 
0.203  0.179 
0.196  0.107 
0.557  0.262 
0.250  0.168 
0.521  0.427 
0.383  0.136 
0.165  0.362 
0.413  0.479 
0.556 


0.306  0.329-0.079-0.174-0.254  0.372  55.556 

0.513  0.398-0.152-0.097-0.172  0.420  56.569 

0.237  0.074-0.155-0.087-0.178  0.205  54.361 

0.272  0.212-0.108-0.157-0.262  0.337  54.292 

0.061  0.348-0.019-0.216-0.264  0.116  55.486 

0.052  0.232-0.251-0.200-0.336  0.192  54.653 

0.388  0.491-0.260-0.471-0.566  0.332  51.431 

0.103  0.175-0.229-0.173-0.193  0.435  60.250 

0.375  0.426-0.263-0.286-0.485  0.239  56.083 

0.237  0.257-0.135-0.211-0.309  0.344  51.278 

0.302  0.408-0.173-0.027-0.094  0.262  0.699 

0.384  0.385-0.224-0.250-0.371  0.306  0.731 

0.399  0.406-0.366-0.396-0.521  0.242  0.795 

0.498  0.448-0.396-0.255-0.378  0.228  0.679 

0.383-0.122-0.196-0.322  0.249  0.605 

-0.430-0.369-0.450  0.336  0.770 

0.471  0.427-0.163  1.802 

0.825-0.1102696.350 
-0.2033549.570 
84.525 
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SD: 

6.375 

5.574 

4.098 

5.327 

5.998 

6.420 

8.388 

4.131 

7.868 

8.021 

0.107 

0.150 

0.116 

0.177 

0.232 

0.133 

0.552 

336.942 

468.301 

4.749 


Table  A-3 


DATA  FILE:  JS02.DAT 
AE  SCHOOL  STATISTICS 

INTERCORRELATIONS,  MEANS  AND  STANDARD  DEVIATIONS 
FOR  20  VARIABLES  IN  A  SAMPLE  OF  173  OBSERVATIONS. 

2345  6789  10 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 


0.295  0.571  0.390  0.137  0.045  0.477  0.194  0.433  0.485 

0.386  0.350  0.205  0.139  0.332  0.405  0.321  0.301 

0.520  0.150  0.089  0.411  0.291  0.304  0.430 

0.179  0.162  0.263  0.270  0.259  0.340 

0.488  0.003  0.221-0.131-0.041 
0.046  0.163-0.041-0.109 
0.240  0.502  0.590 
0.205  0.264 
0.483 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 


11 

12 

13 

14 

15 

16 

17 

13 

19 

20 

MEAN: 

S'D: 

0. 

254 

0. 

303 

0 

.300 

0 

.230 

0 

.246 

0 

.144- 

-0 

.276- 

-0 

.276' 

-0 

.283 

0 

.363 

53 

.561 

6.306 

0. 

346 

0. 

373 

0 

.412 

0 

.320 

0 

.273 

0 

.402- 

-0 

.087- 

-0 

.  151- 

-0 

.096 

0 

.331 

52, 

.399 

6.547 

0. 

133 

0. 

206 

0 

.238 

0 

.020 

0 

.218 

0 

.155- 

-0 

.043- 

-0 

.  183- 

-0 

.138 

0 

.332 

52, 

.376 

5.033 

0. 

189 

0. 

177 

0 

.177 

0 

.043 

0 

.184 

0 

.175- 

-0 

.084- 

-0 

.179- 

-0 

.116 

0 

.240 

52, 

.295 

6.781 

0. 

154 

0. 

102- 

-0 

.024- 

-0 

.164 

0 

.005 

0 

.215- 

-0 

.048- 

-0 

.078- 

-0 

.019 

0 

.093 

54. 

.249 

6.004 

0. 

229 

0. 

129 

0 

.074 

0 

.020- 

-0 

.002 

0 

.359- 

-0 

.014- 

-0 

.  118- 

-0 

.179 

0 

.159 

51. 

,884 

6.328 

0. 

084 

0. 

318 

0 

.293 

0 

.256 

0 

.186 

0 

.122- 

-0 

.187- 

-0 

.211- 

-0 

.151 

0 

.386 

52. 

,387 

7.809 

0. 

152 

0. 

250 

0 

.242 

0 

.125 

0 

.252 

0 

.252 

0 

.002- 

-0 

.083- 

-0 

.021 

0 

.319 

56. 

,052 

5.317 

0. 

199 

0. 

319 

0 

.340 

0 

.  385 

0 

.371 

0 

.  199- 

-0 

.  197- 

-0, 

,  .107- 

-0, 

.  356 

0 

.294 

53. 

,960 

7.178 

0. 

148 

0. 

371 

0, 

,373 

0 

,  309 

0 

,316 

0 

.  1 28- 

-0, 

.  1 34- 

-0, 

,  187- 

-0, 

.  156 

0 

.  40(5 

52. 

,087 

8.082 

0. 

495 

0, 

.369 

0, 

.389 

0, 

,345 

0 

.637- 

-0, 

.265- 

-0, 

,401- 

-0, 

.401 

0 

.317 

0, 

,651 

0.142 

0, 

.493 

0, 

.519 

0, 

,362 

0, 

.519- 

-0, 

,220- 

-0, 

,369- 

-0, 

,315 

0 

.362 

0. 

680 

0.162 

0, 

.450 

0. 

,448 

0, 

,452- 

■0. 

,170- 

-0, 

,310- 

-0. 

,345 

0, 

.393 

0. 

748 

0.118 

0. 

,463 

0, 

.419- 

■0, 

,268- 

■0. 

339- 

-0. 

326 

0, 

.320 

0. 

628 

0.180 

0, 

,310- 

-0, 

137- 

-0. 

421- 

-0. 

374 

0, 

,309 

0. 

478 

0.248 

- 

■0. 

251- 

•0. 

400- 

-0. 

,363 

0, 

,370 

0. 

709 

0.168 

0. 

333 

0. 

374- 

-0, 

,236 

1. 

854 

0.637 

0.663-0.3062784.304  389.963 
-0.2063675.108  485.639 
83.443  5.946 
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Table  A-4 


DATA  FILE:  JS03.DAT 
AMS  SCHOOL  STATISTICS 

INTERCORRELATIONS,  MEANS  AND  STANDARD  DEVIATIONS 
FOR  20  VARIABLES  IN  A  SAMPLE  OF  244  OBSERVATIONS. 

234  56789  10 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 


0.232  0.498  0.267  0.042  0.033  0.144  0.328  0.201  0.328 

0.310  0.217  0.327  0.169-0.124  0.514  0.154  0.001 

0.480  0.134  0.142  0.150  0.289  0.211  0.311 

0.183  0.142-0.010  0.186  0.145  0.128 

0.499-0.093  0.432  0.058  0.045 

-0.147  0.201  0.101-0.039 
-0.120  0.028  0.381 
0.163  0.066 
0.221 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 


12  13  14  15  16  17  18  19  20  MEAN:  SD: 


0.002  0.191  0.091  0.112  0.132-0.047-0 
0.327  0.255  0.329  0.251  0.169  0.300-0 
0.032  0.156  0.115  0.139  0.158-0.024-0 
0.052  0.096  0.050  0.036  0.034  0.032-0 
0.219  0.135  0.213  0.192  0.022  0.322-0 
0.167  0.107  0.129  0.217  0.010  0.182-0 
-0 . 1 59-0 . 098-0 . 078-0 . 054-0 . 083-0 . 223  0 
0.240  0.299  0.326  0.317  0.242  0.311-0 

0.097  0.235  0.259  0.231  0.301  0.150-0 

-0.149  0.055-0.016  0.041  0.056-0.167-0 

0.350  0.320  0.265  0.173  0.520-0 

0.428  0.338  0.265  0.353-0 

0.463  0.386  0.413-0 
0.366  0.389-0 
0.216-0 
-0 


.081-0.029-0.011  0.384  54.471  6.285 

.006-0.050-0.085  0.300  56.004  5.699 

.006  0.012-0.014  0.424  53.471  4.612 

.007-0.035-0.013  0.231  53.361  6.877 

.023-0.217-0.139  0.247  53.201  6.122 

.085-0.110-0.148  0.270  52.295  6.663 

.011  0.094  0.192  0.144  60.139  5.255 

029-0.205-0.172  0.373  54.053  6.952 

035-0.123-0.161  0.231  60.324  5.179 

004  0.016  0.028  0.255  55.197  6.402 

068-0.188-0.300  0.091  0.684  0.115 

195-0.243-0.260  0.171  0.716  0.154 

097-0.168-0.250  0.199  0.806  0.098 

277-0.252-0.251  0.201  0.680  0.186 

105-0.209-0.214  0.117  0.609  0.237 

107-0.276-0.307  0.065  0.755  0.158 

0.352  0.276-0.037  1.776  0.507 

0.692-0.0512728.066  325.099 
-0.0473566.997  445.392 
•  83.515  4.225 


A-4 


Table  A-5 


DATA  FILE:  JS04.DAT 
AO..  SCHOOL  STATISTICS 

INTERCORRELATIONS,  MEANS  AND  STANDARD  DEVIATIONS 
FOR  20  VARIABLES  IN  A  SAMPLE  OF  233  OBSERVATIONS. 


2 


4  5  6  7  8  9  10 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 


0.204  0.498  0.361-0.021  0.021  0.257  0.228  0.386  0.279 
0.317  0.272  0.307  0.266  0.158  0.515  0.350  0.116 
0.507-0.074  0.040  0.269  0.177  0.347  0.302 
0.066  0.116  0.161  0.148  0.222  0.293 
0.689-0.109  0.341-0.034-0.102 
-0.006  0.266  0.074  0.033 
-0.042  0.426  0.512 
0.221-0.027 
0.339 


MEAN: 


0.085 

0.281 

0.007 

0.137 

0.139 

0.188 

-0.018 

0.198 

0.144 

-0.022 


0.169 

0.345 

0.125 

0.114 

0.060 

0.030 

0.140 

0.266 

0.377 

0.192 

0.335 


0.173 

0.254 

0.102 

0.157 

0.039 

0.110 

0.300 

0.245 

0.409 

0.226 

0.313 

0.451 


0.121 

0.282 

0.100 

0.095 

0.028 

0.112 

0.313 

0.220 

0.451 

0.223 

0.303 

0.432 

0.558 


0.213 

0.338 

0.133 

0.180 

0.091 

0.138 

0.164 

0.253 

0.386 

0.113- 

0.297 

0.327 

0.442 

0.509 


0.007-0 
0.359  0 
-0.037-0 
0.068-0 
0.296-0 
0.308-0 
0.089-0 
0.263-0, 
0.230-0. 
-0.046-0, 
0.519-0. 
0.345-0. 
0.345-0. 
0.419-0. 
0.360-0. 
-0. 


.230  0 
.018-0 
,019-0. 
,032  0, 
,078-0. 
,093-0, 
101-0. 
034-0. 
128-0. 
065  0. 
205-0. 
045-0. 
180-0. 
186-0. 
132-0. 
237-0. 
0. 


.005  0 
.041-0 
.024  0 
.031-0 
.121-0, 
.146-0, 
.068-0, 
.018  0, 
.171-0. 
.086  0. 
.190-0. 
.045-0. 
,197-0. 
,186-0. 
188-0. 
254-0. 
125  0. 
0. 


.014  0 
.037  0 
.050  0 
.033  0, 
.107  0, 
.141  0, 
,018  0. 
,016  0. 
,166  0. 
,065  0. 
246  0. 
112  0. 
220  0. 
297  0. 
245  0. 
318  0. 
209-0. 
724-0. 
-0. 


.222  53, 

.313  51, 

.199  51. 

.200  51. 

.230  52. 

.263  50. 

.179  53. 

.390  52. 

.189  53. 

.182  52. 
.060  0. 


237  0.' 

209  0.. 

166  0.1 
146  1.: 

1212770.' 
0483656.! 
85.! 


.107  6.064 
.378  6.508 
.974  5.027 
.755  5.796 
.412  7.036 
,914  6.858 
.335  7.473 
,489  6.788 
,841  7.024 
996  6.274 
653  0.135 
655  0.170 
739  0.133 
603  0.185 
437  0.236 
693  0.185 

836  0.572 
747  370.772 
975  466.868 

837  5.480 
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Table  A- 6 


DATA  FILE:  JS05.DAT  . 

AVSCaOOL  STATISTICS 

INTERCOREELATIONS ,  MEANS  AND  STANDARD  DEVIATIONS 
FOR  20  VARIABLES  IN  A  SAMPLE  OF  197  OBSERVATIONS. 

23  4  56789  10 

1  0.065  0.342  0.181-0.118  0.017  0.135  0.133  0.185  0.178 

2  0.193  0.200  0.270  0.232-0.052  0.364  0.157-0.140 

3  0.287-0.012  0.075  0.049  0,093  0.067  0.060 

4  0.067  0.128-0.028  0.193  0.036-0.096 

5  0.522-0.079  0.185-0.116-0.100 

6  -0.041  0.146-0.075-0.100 

7  -0.154  0.314  0.458 
'  0.100-0.110 


10 


11  12  13  14  15  16  17  18  19 


20  MEAN: 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 


0.097  0.103 
0.406  0.331 
0.119  0.149 
0.153  0.099- 
0.137-0.027 
0.160  0.059 
-0.191  0.022 
0.239  0.178 
0.093  0.197 
-0.215  0.121 
0.389 


0  040  0.071  0.093-0.029-0.148-0.018-0.053  0.113  59.244 

0.453  0.361  0.285  0.386-0.100-0.151-0.193  0.357  58.279 

0.035-0.026  0.076-0.041-0.028  0.040-0.010  0.161  56.117 

0.005  0.156  0.164  0.178  0.067  0.069  0.072  0.163  55.406 

0.050  0.064  0.065  0.138-0.015-0.146-0.111  0.118  54.766 

0  108  0.233  0.138  0.147-0.093-0.135-0.094  0.135  53.523 

0.101-0.035  0.035-0.118-0.105-0.145-0.219  0.085  57.746 

0.268  0.329  0.260  0.346-0.083-0.036  0.000  0.334  59.274 

0.195  0.223  0.313  0.152-0.153-0.238-0.333  0.033  60.437 

0.047-0.107  0.039-0.123-0.056-0.170-0.139  0.167  59.751 

0.335  0.365  0.311  0.490-0.139-0.120-0.156  0.196  0.737 

0.395  0.376  0.345  0.392-0.146-0.199-0.212  0.213  0.764 

0  535  0.418  0.409-0.276-0.331-0.339  0.270  0.834 

0.421  0.480-0.235-0.209-0.256  0.224  0.743 

0.398-0. lb6-0. 222-0. 269  0.157  0.664 

-0.116-0.251-0.216  0.251  0.793 

0.306  0.345-0.100  1.681 

0.775-0.0892630.754 
-0.1463451.591 
89.912 


Sb: 

4.546 

4.676 

3.495 

4.626 

5.924 

6.647 

6.738 

5.137 

5.610 

5.749 

0.124 

0.115 

0.107 

0.147 

0.229 

0.141 

0.472 

306.586 

407.672 

4.173 
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Table  A-7 


DATA  FILE:  JS06.DAT  . 

EM  SCHOOL  DATA 

INTERCOFRELATIONS,  MEANS  AND  STANDARD  DEVIATIONS 
FOR  20  VARIABLES  IN  A  SAMPLE  OF  805  OBSERVATIONS. 

2345  6789  10 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 


0.215  0.554  0.340-0.091  0.020  0.406  0.039  0.423  0.401 
0.300  0.281  0.203  0.228  0.226  0.224  0.339  0.237 
0.430-0.051  0.082  0.336  0.092  0.380  0.333 
0.123  0.162  0.181  0.136  0.255  0.217 
0.552-0.027  0.225-0.025-0.077 
0.036  0.193  0.044  0.021 
-0.044  0.448  0.496 
0.160  0.109 
0.448 


11  12 

1  0.078  0.180 

2  0.343  0.370 

3  0.159  0.248 

4  0.143  0.185 

5  0.176  0.102 

6  0.260  0.190 

7  0.103  0.242 

8  0.226  0.229 

9  0.245  0.376 

10  0.098  0.153 

11  0.399 

12 

13 

14 

15 

16 

17 

18 

19 

20 


13 

14 

15 

0.227 

0.209 

0.260 

0.241 

0.271 

0.302 

0.188 

0.151 

0.223 

0.085 

0.112 

0.149 

0.014 

0.066 

0.027 

0.114 

0.149 

0.091 

0.244 

0.263 

0.281 

0.180 

0.191 

0.199 

0.375 

0.403 

0.382 

0.204 

0.214 

0.258 

0.366 

0.419 

0.352 

0.457 

0.437 

0.430 

0.499 

0.401 

0.441 

16  17  18  19  20  MEAN: 

0.043-0.151-0.107-0.193  0.229  50.966 

0.349-0.112-0.171-0.183  0.331  52.457 

0.063-0.171-0.170-0.206  0.212  50.981 

0.121-0.055-0.077-0.121  0.170  51.867 

0.191-0.103-0.199-0.167  0.114  53.739 

0.260-0.147-0.165-0.156  0.186  51.565 

0.107-0.198-0.154-0.208  0.194  50.108 

0.271  0.002-0.073-0.113  0.255  55.554 

0.242-0.192-0.207-0.278  0.276  52.407 

0.109-0.009-0.008-0.088  0.234  49.822 
0.517-0.215-0.299-0.306  0.161  0.672 

0.394-0.229-0.292-0.315  0.230  0.657 

0.361-0.241-0.275-0.312  0.204  0.744 

0.437-0.301-0.278-0.338  0.180  0.599 

0.332-0.165-0.226-0.246  0.214  0.477 

-0.180-0.279-0.309  0.242  0.714 

0.324  0.337-0.009  1.948 

0.711-0.1012778.937 
-0.1323691.226 
87.904 


A-7 


SD: 

6.666 
6.209 
5.291 
5.964 
6.615 
6.733 
7.770 
5.016 
7.166 
7.735 
0.128 
0.179 
0.123 
0. 176 
0.233 
0.166 
0.700 
399.433 
457.474 
4.730 


Table  A-8 


data  FILE;  JS07.DAT  . 

EN  SCHOOL  STATISTICS 

INTERCORBELATIONS ,  MEANS  AND  STANDARD  DEVIATIONS 
FOR  20  VARIABLES  IN  A  SAMPLE  OF  781  OBSERVATIONS. 

23  456789  10 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 


0.395  0.574  0.404-0.009  0.050  0.410  0.305  0.497  0.488 
0.378  0.297  0.274  0.278  0.259  0.504  0.438  0.306 
0.452-0.015  0.104  0.320  0.225  0.373  0.366 
0.127  0.175  0.208  0.250  0.268  0.264 
0.489-0.145  0.308-0.028-0.052 
0.004  0.262  0,124  0.081 
-0.153  0.443  0.508 
0.250  0.135 
0.480 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 


11  12  13  14  15 


0.194  0.320 
0.433  0.469 
0.210  0.322 
0.191  0.259 
0.150  0.060 
0.230  0.152 
0.061  0.165 
0.329  0.360 
0.254  0.378 
0.128  0.245 
0.483 


0.328  0.321  0.347 
0.397  0.331  0.381 
0.237  0.220  0.296 
0.196  0.160  0.207 
0.016-0.011  0.002 
0.092  0.099  0.106 
0.223  0.225  0.316 
0.320  0.250  0.246 
0.435  0.418  0.433 
0.280  0.241  0.296 
0.391  0.425  0.396 
0.481  0.472  0.406 
0.547  0.474 
0.461 


16 

0.179- 

0.403- 

0.155- 

0.155- 

0.155- 

0.235- 

0.053- 

0.353- 

0.257- 

0.104 

0.602 

0.485 

0.433 

0.441 

0.380 


17 

0.197- 

0.150- 

0.110- 

0.075- 

0.064- 

•0.126- 

-0.152- 

-0.083- 

-0.210- 

-0.075- 

0.194- 

■0.186- 

-0.202- 

•0.263 

-0.152 

-0.218 


18 

-0.091- 

-0.182- 

•0.067- 

-0.111- 

■0.128- 

-0.129- 

-0.115- 

-0.077- 

-0.268- 

-0.097- 

-0.260- 

-0.230- 

-0.223- 

-0.239- 

-0.230- 

-0.286- 

0.349 


19 

•0.177 

•0.262 

-0.149 

•0.115 

■0.100 

•0.162 

-0.144 

-0.130 

-0.320 

-0.121 

-0.272 

-0.261 

-0.267 

■0.282 

-0.306 

-0.288 

0.379 


20 

MEAN: 

SD: 

0.417 

49.881 

7.244 

0.421 

50.703 

6.948 

0.344 

50.859 

5.339 

0.316 

51.261 

6.304 

0.096 

52.026 

7.105 

0.168 

50.851 

6.500 

0.390 

56.223 

6.986 

0.288 

52.087 

6.465 

0.421 

52.741 

7.829 

0.421 

51.298 

7.611 

0.188 

0.649 

0.136 

0.301 

0.611 

0.205 

0.312 

0.737 

0.126 

;  0.267 

0.600 

0.184 

;  0.323 

0.464 

0.236 

;  0.219 

0.677 

0.186 

1-0.162 

1.984 

0.659 

'-0.1492783.830 

393.784 

-0.1953693.070 

453.354 

84.841 

4.925 
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Table  A-9 


DATA  FILE:  JS08.DAT  . 

FC  SCHOOL  STATISTICS 

INTERCOERELATIONS,  MEANS  AND  STANDARD  DEVIATIONS 
FOR  20  VARIABLES  IN  A  SAMPLE  OF  727  OBSERVATIONS. 


234  5  6  789  10 

1  0.032  0.412  0.239-0.037-0.050  0.152  0.067  0.221  0.131 

2  0.144  0.158  0.238  0.218  0.122  0.273  0.186-0.007 

3  0.415-0.053-0.019  0.126  0.033  0.103  0.129 

4  -0.021  0.072  0.081  0.067  0.014  0.108 

5  0.543-0.0720.158-0.043-0.116 

6  -0.039  0.151  0.032-0.064 

"7  -0.195  0.359  0.450 

S  0.104-0.208 

9  0.134 

10 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 


11  12  13  14  15  16  17  18  19  20  MEAN: 

-0.021  0.128  0.086  0.108  0.113  0.002-0.068-0.085-0.105  0.110  58.777 

0.285  0.260  0.277  0.156  0.244  0.344-0.101-0.080-0.091  0.266  57.309 

0.040  0.105  0.031  0.015  0.088  0.009-0.018-0.003-0.050  0.136  55.743 

0.069  0.052  0.049  0.059  0.107  0.051-0.075-0.000-0.049  0.185  55.103 

0.113-0.000  0.019  0.045  0.010  0.118-0.052-0.155-0.116  0.116  53.982 

0.156  0.048  0.090  0.146  0.100  0.193-0.112-0.244-0.239  0.174  52.779 

-0.050  0.070  0.060  0.137  0.184-0.001  0.035  0.025-0.035  0.181  58.344 

0.183  0.172  0.181  0.066  0.102  0.227-0.039-0.110-0.100  0.254  58.087 

0.046  0.197  0.326  0.275  0.316  0.180-0.063-0.210-0.215  0.230  59.564 

-0.070-0.045-0.030  0.049  0.067-0.100  0.031  0.061  0.013  0.175  59.083 
0.333  0.296  0.287  0.294  0.510-0.201-0.280-0.230  0.098  0.724 

0.362  0.319  0.296  0.349-0.169-0.209-0.194  0.173  0.754 

0.420  0.348  0.438-0.160-0.240-0.218  0.203  0.819 

0.386  0.432-0.231-0.227-0.249  0.185  0.714 

0.369-0.118-0.242-0.249  0.211  0.646 

-0.181-0.312-0.277  0.155  0.774 

0.262  0.235  0.008  1.725 

0.731-0.0682623. 172 
-0.1413462.368 
83.491 


A-9 


SD: 

4.678 

4.836 

3.926 

4.563 

6.217 

6.649 

6.393 

5.409 

6.150 

5.656 

0.122 

0.130 

0.101 

0.156 

0.235 

0.142 

0.509 

266.176 

393.828 

5.323 


Table  A-10 


DATA  FILE:  JS09.DAT  . 

GMG  SCHCXDL  STATISTICS 

INTERCORRELATIONS,  MEANS  AND  STANDARD  DEVIATIONS 
FOR  20  VARIABLES  IN  A  SAMPLE  OF  393  OBSERVATIONS. 

234  56789  10 

1  -0.010  0.410  0.320-0.121-0.015  0.303-0.007  0,305  0.229 

2  0.126  0.150  0.201  0.158  0.026  0.239  0.199-0.151 

3  0.449-0.019  0.052  0.216  0.016  0.171  0.143 

4  0.020  0.202  0.083  0.016  0.145  0.077 

5  0.484-0.174  0.335-0.084-0.169 

6  -0.036  0.207  0.035-0.027 

7  -0.250  0.455  0.515 

8  0.027-0.234 

9  0.372 
10 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 


12  13  14  15  16  17  18  19  20  MEAN: 


0.053  0.052  0.142  0.155  0.165  0.065-0 
0.281  0.177  0.175  0.149  0.236  0.233-0 
0.086  0.118  0.135  0.127  0.188  0.074-0 
0.099  0.117  0.164  0.174  0.143  0.121-0 
0.154  0.035-0.001-0.015-0.027  0.201-0 
0.221  0.074  0.147  0.187  0.134  0.239-0 

0.023  0.014  0.151  0.208  0.233-0.016-0 

0.290  0.197  0.160  0.163  0.140  0.312-0 

0.179  0.288  0.380  0.339  0.333  0.175-0 

-0.124-0.009  0.085  0.160  0.125-0.121-0 
0.278  0.385  0.323  0.298  0.514-0 

0.436  0.394  0.374  0.341-0 

0.541  0.406  0.427-0 
0.468  0.376-0 
0.297-0 
-0 


,117-0.138-0.1520.105  54.697  5.391 

.002-0.130-0.117  0.286  52.761  5.428 

.064-0.061-0.052  0.236  53.117  4.451 

.095-0.047-0.082  0.157  52.941  5.665 

.093-0.069-0.051  0.103  53.254  6.485 

.191-0.173-0.198  0.126  51.176  6.788 

.025-0.098-0.1830.189  53.384  7.915 

192- 0.122-0.1240.220  54.463  5.868 

175-0.237-0.338  0.212  54.682  6.555 

013-0.031-0.123  0.108  54.354  6.247 

118-0.292-0.267  0.153  0.689  0.121 

154-0.259-0.280  0.203  0.676  0.170 

273-0.391-0.358  0.194  0.764  0.118 

238-0.288-0.362  0.195  0.631  0.177 

104-0.329-0.350  0.228  0.534  0.239 

193- 0.384-0.397  0.258  0.721  0.168 

0.249  0.290-0.018  1.745  0.544 

0.717-0.1622669.980  327.026 
-0.1813563.469  423.242 
85.965  4.822 


A-10 


Table  A- 11 


DATA  FILE:  JS10.DAT  . 

IIH  (H)  SCHOOL  STATISTICS 

INTERCORRELATIONS,  MEANS  AND  STANDARD  DEVIATIONS 
FOR  20  VARIABLES  IN  A  SAMPLE  OF  554  OBSERVATIONS. 

2345  6789  10 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 


0.469  0.683  0.530  0.130  0.088  0.379  0.462  0.536  0.550 
0.462  0.449  0.434  0.296  0.211  0.679  0.466  0.379 
0.624  0.153  0.146  0.341  0.371  0.439  0.511 
0.245  0.206  0.237  0.341  0.375  0.367 
0.595-0.086  0.477  0.064  0.064 
-0.069  0.364  0.054  0.058 
0.048  0.427  0.556 
0.431  0.339 
0.531 


11 

12 

13 

14 

15 

16 

17 

18 

19 

20 

MEAN: 

SD: 

1 

0.238 

0. 

403 

0. 

390 

0 

.320 

0. 

329 

0. 

220- 

-0 

.258- 

-0 

.138- 

-0 

.214 

0. 

147 

53. 

446 

8. 

058 

2 

0.455 

0. 

536 

0. 

449 

0 

.380 

0. 

444 

0. 

466- 

-0 

.222- 

-0 

.221- 

-0 

.228 

0. 

151 

53. 

240 

7. 

072 

3 

0.208 

0. 

353 

0. 

290 

0 

.275 

0. 

311 

0. 

196- 

-0 

.164- 

-0 

.147- 

-0 

.216 

0. 

162 

53. 

282 

6. 

000 

4 

0.238 

0. 

377 

0. 

249 

0 

.225 

0. 

256 

0. 

218- 

-0 

.154- 

-0 

.154- 

-0 

.216 

0. 

181 

53. 

103 

6. 

128 

5 

0.299 

0. 

273 

0. 

193 

0 

.200 

0. 

179 

0. 

322- 

-0 

.205- 

-0 

.131- 

-0 

.100 

0. 

039 

53. 

473 

6. 

744 

6 

0.317 

0. 

213 

0. 

184 

0 

.247 

0. 

179 

0. 

276- 

-0 

.169- 

-0 

.145- 

-0 

.157 

0. 

050 

53. 

038 

6. 

533 

7 

0.057 

0. 

107 

0. 

157 

0 

.201 

0. 

178 

0. 

071- 

-0 

.008- 

-0 

.091- 

-0 

.131 

0. 

183 

54. 

316 

7. 

374 

8 

0.431 

0. 

494 

0. 

452 

0 

.395 

0. 

441 

0. 

442- 

-0 

.291- 

-0 

.187- 

■0 

.202 

0. 

190 

53. 

051 

8. 

302 

9 

0.267 

0. 

445 

0. 

461 

0 

.428 

0. 

463 

0. 

334- 

-0 

.249- 

-0 

.252- 

-0 

.296 

0. 

226 

55. 

897 

7. 

182 

10 

0.189 

0. 

279 

0. 

281 

0 

.329 

0. 

272 

0. 

186- 

-0 

.119- 

-0 

.125- 

-0 

.190 

0. 

201 

52. 

771 

7. 

993 

11 

0. 

480 

0. 

405 

0 

.370 

0. 

392 

0. 

525- 

-0 

.225- 

-0 

.234- 

-0 

.271 

0. 

137 

0. 

690 

0. 

137 

12 

0. 

458 

0 

.422 

0. 

461 

0. 

469- 

-0 

.193- 

-0 

.199- 

-0 

.229 

0. 

127 

0. 

651 

0. 

205 

13 

0 

.554 

0. 

500 

0. 

429- 

-0, 

.272- 

-0 

.347- 

-0 

.286 

0. 

162 

0. 

756 

0. 

132 

14 

0. 

482 

0. 

490- 

-0 

.248- 

-0 

.300- 

-0 

.332 

0. 

221 

0. 

628 

0. 

201 

15 

0. 

441- 

-0 

.227- 

-0 

.282- 

-0 

.297 

0. 

222 

0. 

508 

0. 

250 

16 

- 

-0 

.253- 

-0 

.330- 

-0 

.286 

0. 

172 

0. 

723 

0. 

185 

17 

n 

.247 

0 

.283- 

-0. 

142 

1. 

827 

0. 

553 

18 

0 

.702- 

■0. 

1612760. 

143 

340. 

312 

19  -0.2043622.607  450.096 

20  1728.251  335.654 


A-11 


Table  A-12 


data  FILE:  JS11.DAT  . 

IIH  (I)  SCHOOL  STATISTICS 

INTERCORRELATIONS,  MEANS  AND  STANDARD  DEVIATIONS 
FOR  20  VARIABLES  IN  A  SAMPLE  OF  320  OBSERVATIONS. 

2345  6789  10 


1  0.505  0.673  0.557  0.140  0.140  0.361  0.476  0.562  0.524 

2  0.483  0.462  0.351  0.281  0.125  0.645  0.503  0.322 

3  0.624  0.169  0.183  0.321  0.349  0.440  0.450 

4  0.136  0.236  0.244  0.332  0.402  0.408 

5  0.599-0.162  0.416  0.077  0.040 

5  -0.0770.3070.1060.119 

7  0.023  0.379  0.527 

g  0.478  0.287 

9  0.520 

10 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 


11 

0.294 

0.444 

0.254 

0.295 

0.245 

0.229 

-0.042 

0.455 

0.384 

0.132 


12  13 

0.391  0.391 
0.510  0.470 
0.393  0.352 
0.384  0.291 
0.201  0.151 
0.201  0.172 
0.112  0.163 
0.474  0.405 
0.470  0.528 
0.267  0.308 
0.514  0.439 
0.517 


14-  15 

0.319  0.408 
0.341  0.428 
0.208  0.379 
0.201  0.323 
0.176  0.184 
0.183  0.204 
0.129  0.226 
0.405  0.491 
0.442  0.498 
0.237  0.379 
0.409  0.380 
0.459  0.505 
0.621  0.509 
0.459 


16  17  18  19 

20 

MEAN: 

SD: 

0.301-0.189-0.162-0.260 

0.227 

53.325 

7.519 

0.432-0.116-0.118-0.179 

0.157 

54.422 

7.149 

0.143-0.176-0.152-0.213 

0.200 

53.566 

5.545 

0.229-0.165-0.146-0.197 

0.185 

53.534 

5.959 

0.207-0.122-0.094-0.071 

0.202 

53.994 

6.507 

0.137-0.094-0.110-0.110 

0.146 

53.572 

6.325 

0.100-0.099-0.078-0.154 

0.101 

54.713 

7.300 

0.471-0.073-0.132-0. 150 

0.180 

53.797 

7.782 

0.416-0. 192-0.247-0.295 

0.244 

56.616 

7.267 

0.136-0.180-0.157-0.201 

0.207 

52.578 

7.732 

0.561-0.117-0.301-0.249 

0.236 

0.709 

0.136 

0 . 487-0 . 1 93-0 . 345-0 . 289 

0.147 

0.686 

0.185 

0.381-0.215-0.320-0.299 

0.235 

0.782 

0.131 

0 . 465-0 . 165-0 . 273-0 . 303 

0.191 

0 . 668 

0.190 

0.464-0.165-0.329-0.346 

0.264 

0.536 

0.256 

-0.109-0.297-0.299 

0.171 

0.762 

0.156 

0.317  0.331- 

-0.210 

1.862 

0.601 

0.753-0.3062744.203  374.216 
-0.3373588.446  454.374 
1734.850  333.830 


A-12 


Table  A- 13 


DATA  FILE:  JS12.DAT  . 

BT/MM  SCHOOL  STATISTICS 

INTERCORRELATIONS,  MEANS  AND  STANDARD  DEVIATIONS 
FOR  20  VARIABLES  IN  A  SAMPLE  OF  837  OBSERVATIONS. 

2345  6789  10 

1  0.397  0.618  0.441  0.043  0.067  0.460  0.216  0.509  0.500 

2  0.379  0.302  0.309  0.243  0.286  0.492  0.467  0.324 

3  0.499-0.015  0.046  0.418  0.166  0.418  0.487 

4  0.089  0.208  0.254  0.183  0.304  0.319 

5  0.557-0.094  0.360  0.007-0.043 

6  0.015  0.250  0.102  0.054 

^  -0.126  0.451  0.545 

^  0.233  0.115 

^  0.474 

10 


11  12  13  14 


15  16  17  13  19  20  MEAN: 


SD: 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 


0.150  0.289  0.320  0.339  0.283  0.190-0.201-0.155-0.203  0.300  50.947  7.674 

0.396  0.442  0.459  0.421  0.381  0.437-0.167-0.184-0.218  0.276  50.836  6.973 

0.180  0.237  0.276  0.253  0.261  0.124-0.171-0.129-0.194  0.186  50.827  5.556 

0.146  0.148  0.170  0.160  0.212  0.115-0.109-0.107-0.163  0.211  51.027  6.083 

0.188  0.079  0.066  0.106  0.028  0.225-0.099-0.104-0.072  0.102  52.896  6.667 

0.238  0.118  0.145  0.188  0.142  0.250-0.167-0.185-0.155  0.149  51.183  6.536 

0.022  0,164  0.262  0.268  0.230  0.048-0.146-0.154-0.197  0.271  54.428  7.323 

0.341  0.316  0.329  0.293  0.306  0,366-0.061-0.085-0. 108  0.191  53.569  6.350 

0.248  0.374  0.434  0.430  0.390  0.296-0.253-0.252-0.289  0.292  52.806  7.577 

0.045  0.162  0.261  0.252  0.226  0.089-0.116-0.096-0.151  0.262  51.375  7.900 

0.417  0.385  0.435  0.374  0.550-0.256-0.266-0.280  0.130  0.660  0.135 

0.453  0.515  0.411  0.443-0.252-0.250-0.302  0.254  0.636  0.191 

0.590  0,460  0.467-0.264-0.268-0.292  0.239  0.741  0.128 

0.490  0.540-0.351-0.312-0.350  0.292  0.605  0.189 

0.400-0.248-0.293-0.325  0.163  0.485  0.237 

-0,275-0.291-0.316  0.180  0.687  0.185 

0.361  0.334-0.127  1.883  0.646 

0.700-0.1572773.877  379.968 
-0.1123677.408  461.697 
82.476  6.495 


A-13 


Table  A- 14 


data  FILE:  JS13.DAT  . 

OS  SCIIOOL  STATISTICS 

INTERCORRELATIONS,  MEANS  AND  STANDARD  DEVIATIONS 
FOR  20  VARIABLES  IN  A  SAMPLE  OF  622  OBSERVATIONS, 

Old  ‘,  6789 


10 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 


0  354  0.559  0.375-0.040-0.139  0.435  0.217  0.492  0.462 
0.319  0.238  0.153  0.026  0.324  0.497  0.487  0.249 
0.447-0.005-0.160  0.366  0.106  0.370  0.342 
-0.049-0.095  0.228  0.051  0.287  0.276 
0.426-0.033  0.073-0.001-0.030 
-0 . 053-0 . 163-0 , 043-0 . 098 
0.122  0.481  0.494 
0.326  0.160 
0.441 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 


11 

0.186 

0.371 

0.151 

0.085 

0.062 

-0.056' 

0.126 

0.335 

0.293 

0.073 


12 

0.298 

0.497 

0.222 

0.178 

0.040 

-0.033 

0.207 

0.396 

0.440 

0.204 

0.459 


13 


14 


15 


16 


0.389  0. 
0.469  0. 
0.288  0. 
0.184  0. 
0.001  0. 
-0.005-0. 
0.327  0. 
0.308  0. 
0.537  0. 
0.304  0. 
0.381  0. 
0.526  0, 
0, 


343  0. 
347  0. 
250  0. 
147  0. 
038-0. 
015  0. 
340 
283 
502 
339 
,376 
,493 
,619 


329  0.214- 
436  0.479- 
278  0.137- 
188  0.071- 
005  0.096- 
004  0.043- 
278  0.133- 
304  0.386- 
0.339- 
0.163- 
.356  0.507' 
.448  0.497 
.431  0.457 
.421  0.468 
0.376 


471 

260 


17 

0.210- 

0.137- 

0.109- 

0.120- 

0.072- 

■0.068- 

■0.159- 

-0.089- 

■0.283- 

-0.163- 

-0.233- 

-0.245 

-0.266 

-0.339 

-0.160 

-0.249 


18 

0.214-' 

0.196- 

0.144- 

•0.105- 

-0.078- 

-0.107- 

■0.227- 

•0.146- 

■0.344- 

■0.194- 

-0.236- 

-0.304- 

-0.303- 

-0.367 

-0.270 

-0.317 

0.303 


19 

0.231 

0,280 

0.179 

0.103 

0.151 

■0,124 

■0.299 

■0.135 

■0.397 

■0.249 

-0.211 

■0.303 

-0.352 

-0.401 

-0.278 

-0.317 

0.290 

0.745 


20 

MEAN; 

SD: 

0.292 

50.021 

7.303 

0.461 

50.170 

6.938 

0.228 

52.611 

4.858 

0.215 

53.121 

4.952 

0.135 

55.892 

5.775 

0.134 

56.929 

5.826 

0.256 

48.929 

7.777 

0.384 

54.777 

5.801 

0.392 

51.195 

7.748 

0.263 

48.902 

7.100 

0.278 

0.687 

0.126 

0.365 

0.647 

0.190 

0.361 

0.736 

0.128 

0.317 

0.593 

0.190 

0.338 

0.473 

0.229 

0.391 

0.709 

0.173 

1-0.139 

1.861 

0.608 

1-0.1902698.407 

337.652 

-0.2503605.212 

458.299 

88.576 

4.500 

A-IA 


Table  A-15 


DATA  FILE:  JS14.DAT  . 

RM  SCHCX)L  STATISTICS 

lOTERCORRELATIONS,  MEANS  AND  STANDARD  DEVIATIONS 
FOR  20  VARIABLES  IN  A  SAMPLE  OF  250  OBSERVATIONS. 


23456  789  10 

1  0.428  0.574  0.433-0.147-0.131  0.463  0.368  0.439  0.492 

2  0.399  0.391  0.229  0.080  0.233  0.524  0.419  0.238 

3  0.550-0.133-0.054  0.355  0.257  0.346  0.418 

4  -0.030-0.120  0.276  0.249  0.301  0.240 

5  0.425-0.038  0.227-0.016-0.152 

^  -0.041  0.029-0.061-0.037 

0.096  0.441  0.613 

8  0.351  0.129 

^  0.430 

10 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 


12  13  14  15  16  17  IS  19  20  MEAN: 

0.134  0.371  0.345  0.252  0.311  0. 146-0.275-0.222-0. ISO  0.370  51.488 

0.352  0.465  0.381  0.234  0.308  0.362-0.164-0.224-0.179  0.423  51.176 

0.088  0.216  0.214  0.147  0.232  0.020-0. 124-0. 088-0. 0S7  0.331  52.184 

0.094  0.187  0.136  0.066  0.176-0.039-0.050-0.049-0.041  0.243  52.948 

0.089  0.005  0.097  0.054-0.009  0.206-0.073-0.049-0.036  0.079  54.640 

0.173  0.074  0.055  0.187-0.010  0.171-0.093-0.149-0.153  0.102  54.800 

-0.021  0.176  0.157  0.212  0.218  0.087-0. 184-0. 168-0.231  0.146  50.828 

0.335  0.419  0.406  0.231  0.273  0.364-0.099-0.066-0.093  0.330  53.640 

0.249  0.501  0.393  0.330  0.413  0.282-0.312-0.301-0.322  0.278  52.504 

0.032  0.250  0.253  0.245  0.287  0.094-0.141-0.185-0.202  0.242  50.136 

0.437  0.389  0.482  0.333  0.572-0.235-0.290-0.312  0.277  0.669 

0.477  0.431  0.409  0.472-0.265-0.335-0.284  0.270  0.633 

0.492  0.441  0.452-0. 318-0. 3S4-0. 373  0.304  0.727 

0.411  0.498-0.348-0.338-0.357  0.220  0.581 

0.431-0.238-0.319-0.287  0.254  0.459 

-0.274-0.363-0.355  0.270  0.696 

0.380  0.320  0.009  1.915 

0.761-0.0962854.634 
-0.0933710.690 
94.715 


SD: 

7.316 
7.435 
5.415 
5.692 
6.691 
5.448 
7.747 
6.632 
7.501 
7.571 
0.137 
0.189 
0.118 
0.192 
0.227 
0.179 
0.652 
416.010 
476. 153 
2.628 
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Table  A- 16 


DATA  FILE:  JS15.DAT  . 

13F  SCHOOL  STATISTICS 

INTERCOREELATIONS,  MEANS  AND  STANDARD  DEVIATIONS 
FOR  20  VARIABLES  IN  A  SAMPLE  OF  819  OBSERVATIONS. 


345  6  789  10 


1  0.456  0.649  0.436  0.035-0.047  0.450  0.375  0.457  0.489 

2  0.444  0.405  0.248  0.095  0.260  0.537  0.340  0.356 

3  0.565  0.039  0.002  0.404  0.304  0.381  0.451 

4  0.129  0.058  0.268  0.281  0.295  0.315 

5  0.516-0.054  0.294-0.067  0.030 

g  -0.054  0.140-0.108-0.009 

7  0.031  0.492  0.512 

o  0.248  0.254 


10 


11  12  13  14  15 

1  0.171  0.322  0.314  0.310  0.309 

2  0.331  0.369  0.403  0.324  0.341 

3  0.152  0.294  0.229  0.275  0.283 

4  0.202  0.263  0.244  0.207  0.192 

5  0.227  0.121  0.110  0.064  0.037 

6  0.187  0.078  0.066  0.078  0.071 

7  0.049  0.159  0.202  0.264  0.255 

8  0.299  0.307  0.352  0.270  0.306 

9  0.157  0.317  0.362  0.402  0.384 

10  0.074  0.215  0.271  0.296  0.291 

11  0.456  0.420  0.387  0.321 

12  0.475  0.461  0.422 

13  0.525  0.479 

14  0.532 

15 

16 

17 

18 

19 

20 


16  17  18  19 

20 

MEAN: 

0.143-0.237-0.178-0.244 

0.327 

53.928 

0 . 322-0 . 1 10-0 . 207-0 .212 

0.404 

55.294 

0.138-0.174-0.168-0.202 

0.324 

53.479 

0.145-0.155-0.129-0.126 

0.345 

53.796 

0.209-0.087-0.125-0.063 

0.189 

54.819 

0.170-0.069-0.107-0.093 

0.210 

54.366 

0.056-0.150-0.150-0.191 

0.294 

53.665 

0.293-0.121-0.175-0.168 

0.347 

56.263 

0.188-0.217-0.231-0.306 

0.323 

56.701 

0.156-0.143-0.196-0.226 

0.297 

52.709 

0.531-0.135-0.309-0.256 

0.331 

0.690 

0.436-0.228-0.309-0.314 

0,373 

0.645 

0.452-0.209-0.322-0.318 

0.396 

0.753 

0.482-0.284-0.366-0.398 

0.384 

0.613 

0.436-0.182-0.303-0.316 

0.377 

0.500 

-0.195-0.373-0.322 

0.335 

0.714 

0.272  0.277- 

-0.196 

1.817 

0.730- 

-0.2712824.191 

-0.2593702.347 

90.410 

SD: 

7.445 

6.283 

5.595 

5.695 

6.190 

6.490 

8.084 

6.896 

6.880 

7.658 

0.142 

0.213 

0.128 

0.205 

0.254 

0.183 

0.580 

439.795 

483.900 

4.127 
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Table  A-17 


DATA  FILE:  JS16.DAT  . 

19K  SCHOOL  STATISTICS 

INTERCORRELATIONS,  MEANS  AND  STANDARD  DEVIATIONS 
FOR  20  VARIABLES  IN  A  SAMPLE  OF  1106  OBSERVATIONS. 


2  3  4  5  6  7  8  9  10 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 


0.452  0.638  0.485  0.069  0.070  0.420  0.442  0.553  0.542 

0.393  0.334  0.349  0.252  0.158  0.633  0.397  0.316 

0.518  0.052  0.102  0.345  0.311  0.432  0.416 

0.112  0.163  0.247  0.303  0.347  0.335 

0.542-0.123  0.350-0.016-0.027 
-0.094  0.260  0.030-0.004 
0.025  0.451  0.571 
0.364  0.245 
0.515 


11  12  13  14  15 

1  0.268  0.390  0.439  0.425  0.399 

2  0.456  0.468  0.446  0.394  0.433 

3  0.216  0.340  0.324  0.312  0.291 

4  0.191  0.285  0.257  0.245  0.225 

5  0.237  0.169  0.124  0.044  0.066 

6  0.238  0.167  0.147  0.139  0.096 

7  0.030  0.153  0.255  0.254  0.228 

8  0.420  0.436  0.463  0.389  0.412 

9  0.269  0.414  0.465  0.449  0.435 

10  0.120  0.255  0.305  0.339  0.301 

11  0.454  0.438  0.427  0.368 

12  0.489  0.491  0.455 

13  0.592  0.494 

14  0.513 

15 

16 

17 

18 

19 

20 


16  17  IS  19  20  MEAN: 

0.283-0.282-0.250-0.291  0.141  52.680 

0.478-0.193-0.242-0.245  0.124  52.802 
0.235-0.201-0.231-0.293  0.067  52.841 

0.188-0.144-0.209-0.205  0.096  53.187 

0.277-0.101-0.157-0.092  0.119  53.599 
0.233-0.135-0.207-0.177  0.113  53.012 

0.064-0.093-0.089-0.180  0.067  54.090 

0.429-0.210-0.251-0.221  0.153  53.103 

0.295-0.222-0.258-0.337  0.105  55.060 

0.122-0.164-0.183-0.256  0.073  52.093 

0.564-0.246-0.295-0.290  0.113  0.682 

0.457-0.216-0.298-0.302  0.052  0.629 

0.480-0.305-0.308-0.327  0.119  0.742 

0.522-0.332-0.305-0.338  0.124  0.610 

0.446-0.258-0.278-0.328  0.084  0.496 

-0.246-0.367-0.343  0.116  0.718 

0.348  0.355-0.084  1.704 

0.710-0.0992741.069 
-0.0833549.440 
1.884 


SD: 

7.593 

7.188 

5.350 

5.783 

6.853 

6.543 

7.873 

7.727 
7.516 

7.728 
0.139 
0.204 
0.130 
0.204 
0.247 
0.186 
0.603 

347.388 

453.307 

0.106 
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Table  A-18 


DATA  FILE;  JS17.DAT  . 

272  (1)  SCHOOL  STATISTICS 

INTERCORRELATIONS,  MEANS  AND  STANDARD  DEVIATIONS 
FOR  20  VARIABLES  IN  A  SAMPLE  OF  484  OBSERVATIONS . 


2  3  4  5  6  7  89  10 

1  0.277  0.484  0.268-0.044-0.023  0.382  0.313  0.434  0.477 

2  0.205  0.113  0.047  0.089  0.170  0.513  0.356  0.256 

3  0.316  0.043  0.062  0.260  0.309  0.333  0.320 

4  0.080  0.176  0.076  0.220  0.163  0.134 

5  0.315-0.096  0.132-0.049-0.115 

6  -0.078  0.184  0.011  0.013 

7  0.046  0.521  0.588 

g  0.336  0.188 

9  0.518 

10 


11 


12  13 


14  15  16 


17  18  19 


20  MEAN: 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 


0  087  0.227  0.291  0.222  0.233  0.039-0.160-0.191-0.237  0.211  55.217 

0.296  0.310  0.357  0.358  0.258  0.299-0.032-0.167-0.193  0.270  58.426 

0  074  0.191  0.196  0.184  0.151  0.081-0.066-0.082-0.140  0.265  55.893 

o!o20  0.046  0.009  0.085  0.057-0.023  0.042-0.026-0.023  0.164  55.979 

0.096  0.064  0.032  0.052-0.002  0.125-0.047-0.094  0.005  0.034  56.339 
0  061  0.015-0.024  0.071-0.019  0.066-0.073-0.001  0.051  0.059  55.866 
-0.009  0.117  0.201  0.227  0.213  0.016-0.224-0.218-0.333  0.218  52.607 

0.318  0.407  0.337  0.373  0.295  0.309-0.010-0.139-0.126  0.251  59.242 

0.151  0.338  0.411  0.418  0.390  0.189-0.231-0.339-0.415  0.293  56.915 

0.044  0.197  0.288  0.239  0.259  0.122-0.162-0.270-0.375  0.261  52.434 

0.325  0.299  0.321  0.197  0.505-0.182-0.131-0.137  0.166  0.762 

0.471  0.473  0.347  0.403-0.063-0.147-0.200  0.177  0.760 

0.513  0.438  0.464-0.196-0.268-0.302  0.220  0.807 

0.413  0.468-0.264-0.263-0.325  0.256  0.683 

0.325-0.194-0.310-0.332  0.273  0.595 

-0.122-0.176-0.180  0.225  0.814 

0.246  0.340-0.168  1.823 

0.744-0.1972783.333 
-0.2013671.466 
83.260 


SD: 

6.451 

5.111 

3.815 

4.357 

6.493 

7.659 

7.737 

6.318 

7.445 

7.598 

0.113 

0.130 

0.112 

0.184 

0.247 

0.126 

0.545 

414.728 

504.741 

5.520 
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Table  A-19 


DATA  FILE:  JS18.DAT  . 

732  SCHOOL  STATISTICS 

INTERCORRELATIONS,  MEANS  AND  STANDARD  DEVIATIONS 
FOR  20  VARIABLES  IN  A  SAMPLE  OF  421  OBSERVATIONS. 

234  5  6789  10 

1  0.323  0.445  0.298-0.110-0.149  0.352  0.204  0.453  0.371 

2  0.164  0.204  0.151  0.073  0.251  0.446  0.398  0.188 

3  0.326-0.062  0.018  0.201  0.124  0.243  0.158 

4  -0.013  0.100  0.095  0.147  0.186  0.116 

5  0.227-0.072  0.234-0.049-0.102 

6  -0.062  0.058-0.141-0.146 

0.072  0.462  0.480 

8  0.305  0.124 

9  0.410 
10 


1 

2 

3 

4 

5 

6 

7 

8 
9 

10 

11 

12 

13 

14 

15 

16 

17 

18 

19 

20 


11 


12 


13 


14 


15 


0.113 

0.287 

0.081 

0.072 

0.106 

0.039 

0.044 

0.226 

0.132 

0.002 


0.173  0 
0.246  0 
0.150  0 
0.100  0 
0.075-0, 
0.009-0, 
0.123  0. 
0.272  0, 
0.295  0. 
0.115  0. 
0.361  0. 
0. 


,210  0 
.319  0 
,126  0 
062  0 
036-0 
043-0 
143  0 
250  0 
351  0 
180  0 
319  0 
400  0 
0 


.176  0.25S 
.270  0.251 
.074  0.185 
.008  0.094 
.020-0.051 
.004-0.009 
.246  0.237 
.216  0.251 
.386  0.355 
.216  0.125 
.346  0.343 
.402  0.343 
,495  0.406 
0.445 


16  17  18  19 

20 

MEAN: 

SD: 

0.056-0.136-0. 132-0.140 

0.327 

51.544 

6.762 

0.395-0.161-0.158-0.245 

0.409 

53.591 

6.149 

0.028-0.031  0.015-0.047 

0.273 

53.183 

5.642 

0 . 056-0 . 043-0 . 064-0 . 029 

0.272 

54.052 

5.063 

0.123  0.066  0.024  0.048 

0.129 

57.278 

5.367 

0.068  0.026-0.009-0.005 

0.091 

57.264 

7.324 

0.064-0.136-0.212-0.314 

0.190 

47.112 

6.736 

0.295-0.033-0.112-0.143 

0.369 

55.713 

6.608 

0 . 246-0 . 264-0 .318-0. 388 

0.252 

50.708 

7.382 

0.044-0.138-0.139-0.217 

0.143 

47.846 

7.553 

0.539-0. 136-0. 166-0. 178 

0.282 

0.715 

0.119 

0.357-0.147-0.188-0.180 

0.287 

0.674 

0.168 

0.421-0.272-0.274-0.326 

0.226 

0.737 

0.121 

0 . 470-0 . 295-0 . 342-0 . 350 

0.175 

0.594 

0.189 

0.348-0. 187-0.236-0.283 

0.270 

0.456 

0.225 

-0.227-0.289-0.285 

0.224 

0.744 

0.154 

0.298  0.275- 

•0.032 

2.057 

0.617 

0.717 

0.0023054.077 

505.946 

-0.0243973.984  487.646 
81.672  6.276 
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Table  A-20 


Combined  Schools  Predictor  Matrix 


Means,  Standard  Deviations,  and  Intercorrelations  for  a 
Combined  Schools  Predictor  Matrix  of  9038  Subjects 
in  Eighteen  Schools 


AR: 


WK: 


PC: 


NO: 


CS: 


AS: 


MK: 


MC: 


El: 


GS: 

AR: 

WK: 

PC: 

NO: 

CS: 

AS: 

MK: 

MC: 

El: 


0.420  0.607  0.428  0.016  0.012  0.407  0.335  0.513  0.509 

0.404  0.346  0.269  0.198  0.228  0.533  0.442  0.308 

0.518  0.044  0.089  0.313  0.271  0.396  0.400 

0.121  0.166  0.185  0.254  0.292  0.276 

0.525-0.123  0.310-0.022-0.058 
-0.090  0.211  0.014-0.029 
-0.022  0.484  0.562 
0,296  0.179 
0.504 


GS: 

AR: 

WK: 

PC: 

NO: 

CS: 

AS: 

MK: 

MC: 

El: 

CT: 

SM: 

FR: 

ID: 

AO: 

SO: 

Tl: 

T2: 

TI: 


CT: 

0.198 

0.429 

0.166 

0.163 

0.219 

0.212 

0.066 

0.373 

0.285 

0.119 


SM: 

0.199 

0.406 

0.200 

0.192 

0.196 

0.212 

0.031 

0.348 

0.241 

0.090 

0.558 


FR:  ID:  AO: 


0.333 

0.352 

0.328 

0.445 

0.424 

0.361 

0.301 

0.270 

0.243 

0.243 

0.201 

0.180 

0.117 

0.072 

0.063 

0.118 

0.094 

0.120 

0.159 

0.241 

0.259 

0.381 

0.357 

0.308 

0.391 

0.450 

0.436 

0.230 

0.286 

0.292 

0.458 

0.456 

0.486 

0.445 

0.398 

0.400 

0.489 

0.474 

0.565 

SO:  Tl:  T2:  TI:  MEAN: 

0.340-0.178-0.229-0.223  52.995 

0.390-0.175-0.208-0.149  53.382 

0.285-0.130-0.181-0.149  52.888 

0.212-0.103-0.127-0.107  53.095 

0.042-0.097-0.068-0.076  54.059 

0.083-0.104-0.108-0.104  53.172 

0.260-0.168-0.223-0.144  53.580 

0.324-0.135-0.139-0.107  54.883 

0.437-0.275-0.339-0.238  54.790 

0.292-0.165-0.219-0.136  52.345 

0.406-0.306-0.307-0.221  0.725 

0.358-0.250-0.254-0.208  0.688 

0.433-0.260-0.274-0.206  0.666 

0.477-0.299-0.318-0.250  0.759 

0.489-0.308-0.347-0.295  0.627 

-0.287-0.318-0.199  0.515 

0.730  0.320  2762.943 

0.334  3638.037 

1.841 


SD: 

7.384 

6.897 

5.375 

5.832 

6.606 

6.876 

8.014 

6.874 

7.669 

7.892 

0.175 

0.134 

0.189 

0.126 

0.192 

0.247 

385.782 

468.824 

0.609 
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Table  A-21 


Population  #2  Matrix: 

Population  Intercorrelations,  Means,  and  Standard 
Deviations  for  the  Combined  ASVAB  and  ECAT  tests. 


AR: 

GS:  0.611 

AR: 

WK: 

PC: 

NO: 

CS: 

AS: 

MK: 

MC: 

El: 


WK:  PC: 

0.720  0.608 
0.596  0.574 
0.732 


NO:  CS: 

0.275  0.249 
0.470  0.395 
0.324  0.328 
0.396  0.386 
0.640 


AS:  MK: 

0.520  0.554 
0.400  0.707 
0.437  0.497 
0.339  0.500 
0.047  0.496 
0.058  0.408 
0.197 


MC:  El: 

0.638  0.625 
0.613  0.487 
0.547  0.534 
0.485  0.444 
0.228  0.145 
0.221  0.147 
0.618  0.669 
0.494  0.370 
0.630 


CT: 

SM: 

FR: 

ID: 

AO: 

GS 

0.370 

0.369 

0.496 

0.499 

0.469 

AR 

0.557 

0.536 

0.590 

0.565 

0.507 

WK 

0.342 

0.372 

0.470 

0.427 

0.395 

PC 

0.349 

0.375 

0.438 

0.385 

0.354 

NO 

0.370 

0.352 

0.306 

0.254 

0.235 

CS 

0.348 

0.348 

0.286 

0.253 

0.263 

AS 

0.206 

0.170 

0.307 

0.377 

0.386 

MK 

0.516 

0.494 

0.540 

0.511 

0.462 

MC 

0.427 

0.385 

0.527 

0.570 

0.553 

El 

0.269 

0.241 

0.383 

0.425 

0.422 

CT 

0.628 

0.555 

0.548 

0.568 

SM 

0.542 

0.497 

0.492 

FR 

0.587 

0.568 

ID 

0.641 

AO 

SO 

T1 

T2 

TI 


SO:  Tl:  T2:  TI:  MEAN: 

0.488-0.283-0.343-0.307  50.615 

0.537-0.291-0.337-0.257  50.664 

0.439-0.243-0.301-0.247  51.311 

0.392-0.226-0.264-0.218  51.156 

0.225-0.206-0.195-0.180  52.512 

0.239-0.200-0.215-0.192  52.266 

0.391-0.256-0.322-0.222  51.409 

0.482-0.254-0.276-0.221  51.210 

0.559-0.365-0.435-0.319  51.941 

0.429-0.262-0.325-0.225  50.333 

0.504-0.374-0.385-0.291  0.681 

0.461-0.323-0.336-0.278  0.657 

0.540-0.341-0.366-0.284  0.612 

0.572-0.373-0.403-0.320  0.723 

0.577-0.378-0.424-0.358  0.577 

-0.361-0.402-0.274  0.448 

0.750  0.362  2818.957 

0.380  3714.969 

1.914 


7 

7 

8 

7 
9 

8 
9 
8 
0 
0 
0 
0 
0 
0 

399 

491 

0 


SD: 

.773 

.645 

.354 

.964 

.013 

.812 

.168 

.689 

.127 

.856 

.191 

.146 

.212 

.140 

.210 

.273 

.981 

.781 

.626 
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Table  A-22 


Matrix  of  Fully  Corrected  School  Validities 

Validities  Corrected  for  Restriction  in  Range  and 
Criterion  Unreliability  for  the  ASVAB  and  ECAT  Tests 
and  School  Sample  Sizes  for  Eighteen  Joint  Service  Schools 


GS:  AR:  WK:  PC:  NO: 

AC:  0.630  0.729  0.527  0.576  0.460 
AE:  0.586  0.579  0.545  0.467  0.318 

AMS:  0.707  0.672  0.696  0.601  0.404 

AO:  0.547  0.588  0.503  0.507  0.405 
AV:  0.611  0.734  0.607  0.578  0.398 
EM:  0.538  0.618  0.488  0.452  0.360 
EN:  0.627  0.632  0.579  0.545  0.326 
FC:  0.610  0.686  0.594  0.589  0.405 
GMG:  0.564  0.679  0.598  0.535  0.367 
IIH-H:  0.248  0.238  0.251  0.267  0.085 
IIH-I:  0.320  0.289  0.295  0.291  0.302 
BT/MM:  0.456  0.441  0.359  0.382  0.244 
OS:  0.590  0.702  0.582  0.597  0.485 

RM:  0.619  0.655  0.591  0.537  0.404 

13F:  0.583  0.677  0.578  0.598  0.426 

19K:  0.191  0.186  0.135  0.160  0.182 

272-1:  0.572  0.644  0.640  0.593  0.345 
732:  0.666  0.753  0.658  0.652  0.467 


CT:  SM:  FR:  ID:  AO: 

AC:  0.540  0.502  0.583  0.479  0.526 
AE:  0.513  0.469  0.501  0.567  0.536 

AMS:  0.421  0.425  0.515  0.540  0.499 
AO:  0.411  0.348  0.469  0.464  0.478 
AV:  0.519  0.484  0.531  0.535  0.502 

EM:  0.447  0.374  0.467  0.463  0.411 
EN:  0.410  0.352  0.486  0.506  0.475 

FC:  0.457  0.429  0.553  0.549  0.520 
GMG:  0.494  0.403  0.527  0.496  0.464 

IIH-H:  0.232  0.202  0.219  0.252  0.297 
IIH-Z:  0.265  0.311  0.216  0.311  0.287 
BT/MM:  0.317  0.279  0.410  0.385  0.417 
OS:  0.562  0.513  0.561  0.539  0.505 

RM:  0.493  0.487  0.470  0.511  0.448 

13F:  0.541  0.522  0.585  0.590  0.565 

19K:  0.158  0.155  0.112  0.157  0.169 

272-1:  0.511  0.456  0.493  0.488  0.491 
732:  0.508  0.550  0.605  0.550  0.492 


CS:  AS:  MK:  MC:  El:  N: 


0.502 

0.345 

0.780 

0.500 

0.508 

72 

0.356 

0.450 

0.564 

0.547 

0.536 

173 

0.434 

0.531 

0.616 

0.652 

0.600 

244 

0.410 

0.358 

0.618 

0.466 

0.464 

233 

0.372 

0.421 

0.684 

0.526 

0.594 

197 

0.350 

0.352 

0.584 

0.518 

0.471 

805 

0.303 

0.576 

0.552 

0.634 

0.615 

781 

0.387 

0.461 

0.657 

0.626 

0.586 

727 

0.332 

0.450 

0.613 

0.564 

0.521 

393 

0.096 

0.291 

0.249 

0.316 

0.283 

554 

0.225 

0.244 

0.270 

0.347 

0.289 

320 

0.243 

0.405 

0.410 

0.458 

0.424 

837 

0.517 

0.364 

0.695 

0.579 

0.480 

622 

0.419 

0.312 

0.572 

0.530 

0.459 

250 

0.465 

0.457 

0.610 

0.608 

0.498 

819 

0.168 

0. 123 

0.201 

0.171 

0.125 

1106 

0.329 

0.435 

0.539 

0.573 

0.511 

484 

0.426 

0.410 

0.690 

0.581 

0.496 

421 

SO: 

Tl: 

T2: 

TI: 

N; 

0.481-0.225-0.280-0.232  72 
0.515-0.362-0.346-0.356  173 
0.493-0.295-0.348-0.285  244 
0.446-0.353-0.317-0.334  233 
0.479-0.258-0.371-0.264  197 
0.438-0.256-0.291-0.179  805 
0.502-0.305-0.359-0.285  781 
0.530-0.265-0.358-0.216  727 
0.491-0.300-0.339-0.189  393 
0.302-0.227-0.272-0.206  554 
0.324-0.364-0.405-0.271  320 
0.326-0.260-0.236-0.207  837 
0.525-0.288-0.356-0.257  622 
0.473-0.207-0.250-0.108  250 
0.571-0.370-0.399-0.322  819 
0.136-0.128-0.125-0.109  1106 
0.537-0.349-0.366-0.342  484 
0.551-0.232-0.266-0.235  421 
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Appendix  B 

ASVAB  and  ECAT  Reliabilities 


B-0 


Table  B-1 


Paper  and  Pencil  ASVAB  Subtest  Reliabilities  and  Standard 
Deviations;  Estimated  from  Statistics  for  ASVAB  Form  9B 


ASVAB  Form  9B 

Standard  Deviations: 

ASVAB  Form  9B 
Reliabilities: 

Sample; 

Population; 

Sample;  Population; 

• 

i: 

Test  i: 

Si: 

Si: 

1 

GS 

9.01 

10.02 

.80 

.84 

2 

AR 

8.39 

9.30 

.82 

.85 

3 

WK 

6.83 

7.88 

.84 

.88 

4 

PC 

7.72 

8.58 

.59 

.67 

5 

NO 

7.04 

8.03 

.82 

.86 

6 

CS 

7.85 

8.59 

.79 

.82 

7 

AS 

8.24 

8.86 

.80 

.83 

8 

MK 

8.38 

9.12 

.85 

.87 

9 

MC 

8.40 

8.98 

.73 

.76 

10 

El 

7.77 

8.13 

.66 

.69 
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Table  B-2 


ECAT  Sample  Standard  Deviations  amd  Reliabilities,  and 
Estimated  Population  Standard  Deviations  and  Reliabilities 


ECAT  Subtest  ECAT  Subtest 

Standard  Deviations:  Reliabilities : 


1: 

Test 

Sample; 

i:  s^: 

Population; 

Sit 

Sample ; 

Population ; 

Riit 

1 

CT 

.160 

.191 

• 

VO 

.85 

2 

SM 

.140 

.146 

.81 

.83 

3 

FR 

.199 

.212 

.75 

• 

09 

4 

ID 

.132 

.140 

.79 

H 

00 

• 

5 

AO 

.214 

.210 

.83 

.82 

6 

SO 

.258 

.273 

.75 

.78 

7 

T1 

432 

399.781 

CD 

• 

.81 

8 

T2 

531 

491.781 

.91 

t 

VO 

o 

9 

TI 

.568 

.626 

.80 

.84 
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Appendix  C 

CAT-ASVAB  Reliabilities, 
Intercorrelations,  and  Validities 


C-0 


Table  C-1 


Number  of  Items,  Average  Completion  Times,  and  Reliability 
Function  Parameters  for  CAT-ASVAB  Power  Tests 


Test  #:  Test: 


Number  of 
'o(i)* 


Average  Completion 


Time,  t 


o(  i )  * 


W, 


1  GS  15 

2  AR  15 

3  WK  15 

4  PC  10 

7  AS  20 

8  MK  15 

9  MC  15 

10  El  15 


4.712 

20.973 

4.112 

13.604 

6.196 

9.461 

10.787 

4.419 


.25813 

.34615 

.34779 

.37594 

.31249 

.58828 

.26707 

.22787 


.77643 

.75918 

.99696 

.98065 

1.08341 

1.15528 

.86810 

1.12103 


1.03190 

1.05059 

1.23127 

1.13405 

1.26922 

1.33384 

1.04683 

1.18516 
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Table  C-2 


ASVAB  and  CAT-ASVAB  Reliabilities,  and  Weights  for 
Estimating  CAT-ASVAB  Intercorrelations 


Test  #:  Test:  ^c(i)* 


W,.: 


1 

GS 

.88 

.84 

2 

AR 

.92 

.85 

3 

WK 

.91 

.88 

4 

PC 

.84 

.67 

7 

AS 

.92 

.83 

8 

MK 

.93 

.87 

9 

MC 

.87 

.76 

10 

EX 

.82 

.69 

1.0235 

1.0404 

1.0169 

1.1197 

1.0528 

1.0339 

1.1447 

1.0901 
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Table  C-3 


Estimated  CAT-ASVAB  Z-Score  Intercorrelations  (Mean  =  0,  SD  =  1) 
for  the  1991  DOD  Population  of  Applicants 


6S: 

AR: 

WK: 

PC: 

NO: 

CS: 

AS: 

MK: 

MC: 

El: 


AR:  WK:  PC: 

0.652  0.749  0.698 
0.631  0.671 
0.833 


NO: 

CS: 

AS: 

MK: 

MC: 

EX: 

MEAN: 

0.282 

0.255 

0.560 

0.588 

0.697 

0.695 

0.000 

0.490 

0.412 

0.439 

0.763 

0.682 

0.551 

0.000 

0.329 

0.333 

0.466 

0.522 

0.593 

0.590 

0.000 

0.444 

0.433 

0.400 

0.580 

0.581 

0.542 

0.000 

0.640 

0.049 

0.514 

0.243 

0.158 

0.000 

0.061 

0.422 

0.236 

0.160 

0.000 

0.214 

0.693 

0.765 

0.000 

0.546 

0.416 

0.000 

0.731 

0.000 

0.000 
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SD: 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


Table  C-4 


Estimated  CAT-ASVAB  Validities,  corrected  for  Criterion  Unreliability 
and  Range  Restriction,  together  with  Sample  Sizes,  in 
in  Eighteen  Joint  Service  Schools 


GS: 

AR: 

WK: 

PC: 

NO: 

CS: 

AS: 

MK: 

MC: 

EI: 

N: 

AC: 

0.645 

0.760 

0.535 

0.646 

0.460 

0.502 

0.363 

0.808 

0.534 

0.553 

72 

A£: 

0.600 

0.604 

0.553 

0.524 

0.318 

0.356 

0.473 

0.585 

0.584 

0.583 

173 

AMS: 

0.724 

0.700 

0.706 

0.674 

0.404 

0.434 

0.558 

0.638 

0.696 

0.653 

244 

AO: 

0.560 

0.613 

0.511 

0.568 

0.405 

0.410 

0.376 

0.640 

0.497 

0.504 

233 

AV: 

0.626 

0.764 

0.617 

0.648 

0.398 

0.372 

0.442 

0.709 

0.561 

0.645 

197 

EM: 

0.551 

0.643 

0.495 

0.507 

0.360 

0.350 

0.370 

0.605 

0.553 

0.512 

805 

EN: 

0.642 

0.658 

0.588 

0.611 

0.326 

0.303 

0.605 

0.571 

0.676 

0.668 

781 

FC: 

0.625 

0.714 

0.603 

0.660 

0.405 

0.387 

0.485 

0.681 

0.668 

0.637 

727 

GMG: 

0.578 

0.707 

0.607 

0.600 

0.367 

0.332 

0.473 

0.635 

0.602 

0.567 

393 

IIH-H: 

0.253 

0.248 

0.255 

0.300 

0.085 

0.096 

0.306 

0.258 

0.337 

0.308 

554 

IIH-I: 

0.328 

0.301 

0.300 

0.327 

0.302 

0.225 

0.257 

0.280 

0.370 

0.314 

320 

BT/MM: 

0.467 

0.459 

0.365 

0.428 

0.244 

0.243 

0.426 

0.424 

0.489 

0.461 

837 

OS: 

0.604 

0.732 

0.591 

0.669 

0.485 

0.517 

0.382 

0.719 

0.618 

0.521 

622 

RM: 

0.634 

0.682 

0.600 

0.603 

0.404 

0.419 

0.328 

0.592 

0.566 

0.499 

250 

13F: 

0.597 

0.705 

0.587 

0.671 

0.426 

0.465 

0.481 

0.632 

0.649 

0.541 

819 

19K: 

0.195 

0.194 

0.138 

0.179 

0.182 

0.168 

0.129 

0.209 

0.182 

0.136 

1106 

272-1: 

0.586 

0.671 

0.650 

0.665 

0.345 

0.329 

0.457 

0.559 

0.612 

0.555 

484 

732: 

0.682 

0.784 

0.668 

0.731 

0.467 

0.426 

0.431 

0.715 

0.620 

0.539 

421 
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Appendix  D 

Time  Limits  and  Estimated  Population  Mean  Completion  Times  for 

Relevant  ASVAB  and  ECAT  Subtests 


D-0 


Table  D-1 


Time  Limits  (paper  &  pencil  testing)  and  Estimated  Population 
Mean  Completion  Times  (computerized  testing)  for  relevant 

ASVAB  and  ECAT*  Subtests^ 


_ Testing  Method: _ 

Subtest:  Paper  and  Pencil:  Computerized: 


ASVAB 

GS:  11.000  4.712 
AR:  36.000  20,973 
WK:  11.000  4.112 
PC:  13.000  13.604 
NO:  3.000  3.000 
CS:  7.000  7.000 
AS:  11.000  6.196 
MK:  24.000  9.461 
MC:  19.000  10.787 
El:  9.000  4.419 

ECAT 

CT:  9.894 
SM:  16.274 
FR:  13.398  9.434 
ID:  14.430 
AO:  16.231  9.778 
SO:  8.593  5.657 
Tl:  3.762 
T2:  3.764 
TI:  2.692 


1.  Only  the  FR,  AO,  and  SO  ECAT  subtests  can  be  admministered  in 

paper  and  pencil  form. 

2.  Times  are  in  minutes. 
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Appendix  E 

Initial  Matrix  Derivation 


E-0 


The  following  steps  illustrate  how  the  matrices  in  each 
contrast  were  generated  from  our  Population  #2  Matrix  and  school 
validity  matrices.  For  simplicity  we  consider  only  four  tests  that 
are  reduced  in  length  by  20%,  and  four  schools. 

We  start  out  with; 


(1)  Original  Test  Intercorrelations; 


1 

2 

3 

4 

GS 

AR 

WK 

PC 

1 

GS 

1.000 

.611 

.720 

.608 

2 

AR 

1.000 

.596 

.574 

3 

WK 

1.000 

.732 

4 

PC 

1.000 

(2) 

Original  Validities; 

1 

2 

3 

4 

GS 

AR 

WK 

PC 

1 

AC 

.614 

.711 

.513 

.561 

2 

AE 

.565 

.559 

.526 

.451 

3 

AMS 

.682 

.648 

.671 

.579 

4 

AO 

.516 

.555 

.474 

.478 

(3)  Original  Reliabilities; 

12  3  4 
GS  AR  WK  PC 
.86  .91  .92  .81 


(4)  The  Reliabilities  for  length  n  =  .8,  shown  below,  were 

generated  from  Spearman-Brown; 


“■Jin 


1+  (n-l)  (r^) 


Thus,  e.g.,  for  GS; 


r 


.8, .8 


.8 (.86) 
l-(.2)  (.86) 


.831 
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Reliabilities  for  Length  n  =  .8: 

1  2  3  4 

GS  AR  WK  PC 

.831  .890  .902  .773 


In  order  to  calculate  the  intercorrelations  and  validities  for 
tests  of  altered  length,  the  following  formulae  apply. 

Let  X(T),  Y(T)  be  "true  scores"  on  tests  X  and  Y;  X(l),  Y(l) 
be  scores  on  tests  X  and  Y  of  unit  (or  original)  length,  and  X(n), 
Y(n)  be  scores  on  tests  X  and  Y  or  altered  length,  n. 

From  the  correction  for  attenuation,  we  know: 


^X(T)  ,  y(T> 


_ ^x(i) ,  r(i) _ 

^^x{i).x{x)  ■^y(i),r(i) 


[1] 


Similarly: 


^x[T\ ,  ym 


•Jf(ii) ,  r(n) 


yj^X{n)  .X(n)  ^y(n),y{n) 


[2] 


From  [1]  and  [2]: 


^X(n) ,  y{n) 


^xu) .  r(i) 


'N 


^Xin) ,  Xln) 
^xa),x{i) 


^y{n) ,  y(n)  ^ 
•^r(i) ,  rd) 


[3] 


or: 


^X(n),  y(n)  ^X(l),  Y(l) 


[4] 


where  the  weights  are  the  square  root  of  the  ratios  of  the  two 
reliabilities  for  the  test  in  question. 


E-2 


similarly,  for  validities,  where  only  X  is  modified  in  length, 
we  have: 

^X(n),  y(l)  “  ^X(l),  Y(l) 


(5)  The  weights  were  generated  from  the  original  and  n  -.8 
reliabilities,  e.g.: 


Weights: 


W1 

.983 


.831  _ 


N  86 


.983 


W2 

.989 


W3 

.990 


W4 

.977 


(6) 


The  intercorrelations  for  n  = 
TesV 'inter correlations  f  or  ^n  = 


.8  were  generated, 
(.611) (.983) (.989) 
.8: 


e.g. , 

=  .594 


1 

2 

3 

4 

GS 

AR 

WK 

PC 

1 

GS 

1.000 

.594 

.701 

.584 

2 

AR 

* 

1.000 

.584 

.555 

3 

WK 

1.000 

.708 

4 

PC 

1.000 

(7) 

The 

validities 

were  generated,  e.g. 

^l{n)MhLX)  ~  ^l(l)»A(l) 

Validities  for  n  »  .8: 

.983)  = 

1 

2  3 

4 

GS 

AR  WK 

PC 

1 

AC 

.604 

.703  .508 

.548 

2 

AE 

.555 

.553  .521 

.441 

3 

AMS 

.670 

.641  .664 

.566 

4 

AO 

.507 

.549  .469 

.467 

604 
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Appendix  F 

Optional  Test  Length  Intercorrelations  and  Validities 
for  Each  Testing  Condition 


F-0 


Table  F-1 


ASVAB  Z-Score 

!  intercorrelations  (Mean  =  0, 

SD  =  1) 

for 

Optimal  Test  Lengths  and  100  Minutes  Testing  Time 

AR:  WK:  PC: 

NO:  CS: 

AS:  MK: 

MC: 

EX: 

MEAN: 

SD: 

GS: 

0.551  0.708  0.483 

0.270  0.244 

0.496  0.525 

0.587 

0.651 

0.000 

1.000 

AR: 

0.547  0.426 

0.432  0.363 

0.356  0.625 

0.527 

0.474 

0.000 

1.000 

WK: 

0.592 

0.324  0.328 

0.423  0.478 

0.513 

0.567 

0.000 

1.000 

PC; 

0.320  0.312 

0.266  0.389 

0.368 

0.381 

0.000 

1.000 

NO: 

0.640 

0.045  0.478 

0.213 

0.154 

0.000 

1.000 

CS: 

0.056  0.393 

0.207 

0.156 

0.000 

1.000 

AS: 

0.183 

0.561 

0.688 

0.000 

1.000 

HK: 

0.445 

0.377 

0.000 

1.000 

MC: 

0.626 

0.000 

1.000 

El: 

0.000 

1.000 

Test: 

Time  (minutes): 

Reliability: 

GS: 

9 

.81 

AR: 

16 

.72 

WK: 

11 

.88 

PC: 

5 

.44 

NO: 

3 

.86 

CS: 

7 

.82 

AS: 

8 

.78 

KK: 

15 

.81 

MC: 

12 

.67 

El: 

14 

.78 

TOTAL: 

100 

Table  F-2 


ASVAB  Validities,  Corrected  for  Criterion  Unreliability  and 
Range  Restriction,  together  with  Sample  Sizes,  in  Eighteen  Joint 
Service  Schools,  with  Optimal  Test  Lengths  and  100  Minutes 

Testing  Time 


GS: 

AR: 

WK: 

AC 

0.619 

0.669 

0.527 

AE 

0.576 

0.532 

0.545 

AMS 

0.695 

0.617 

0.696 

AO: 

0.537 

0.540 

0.503 

AVj 

0.600 

0.673 

0.607 

EM: 

0.529 

0.567 

0.488 

EN: 

0.616 

0.580 

0.579 

FCj 

0.600 

0.629 

0.594 

GMG: 

0.555 

0.623 

0.598 

IIH-H:  0.243 

0.218 

0.251 

11H-] 

::  0.315 

0.266 

0.295 

BT/MM:  0.448 

0.404 

0.359 

OS: 

0.580 

0.644 

0.582 

RM: 

0.609 

0.601 

0.591 

13P: 

0.573 

0.621 

0.578 

19K: 

0.187 

0.171 

0.135 

272-1 

:  0.562 

0.591 

0.640 

732: 

0.655 

0.691 

0.658 

PC:  NO:  CS:  AS: 

0.466  0.460  0.502  0.335 
0.378  0.318  0.356  0.436 
0.486  0.404  0.434  0.515 
0.410  0.405  0.410  0.347 
0.468  0.398  0.372  0.408 
0.366  0.360  0.350  0.341 
0.441  0.326  0.303  0.558 
0.477  0.405  0.387  0.447 
0.433  0.367  0.332  0.436 
0.216  0.085  0.096  0.282 
0.236  0.302  0.225  0.237 
0.309  0.244  0.243  0.393 
0.483  0.485  0.517  0.352 
0.435  0.404  0.419  0.302 
0.484  0.426  0.465  0.443 
0.129  0.182  0.168  0.119 
0.480  0.345  0.329  0.422 
0.528  0.467  0.426  0.397 


MK: 

MC: 

El: 

N: 

0.751 

0.469 

0.539 

72 

0.544 

0.512 

0.569 

173 

0.593 

0.611 

0.637 

244 

0.595 

0.436 

0.492 

233 

0.659 

0.492 

0.630 

197 

0.562 

0.485 

0.500 

805 

0.531 

0.593 

0.652 

781 

0.633 

0.587 

0.621 

727 

0.591 

0.528 

0.553 

393 

0.240 

0.296 

0.300 

554 

0.260 

0.325 

0.307 

320 

0.395 

0.429 

0.449 

837 

0.669 

0.543 

0.509 

622 

0.551 

0.496 

0.487 

250 

0.588 

0.569 

0.528 

819 

0.194 

0.160 

0.133 

1106 

0.520 

0.537 

0.542 

484 

0.664 

0.544 

0.526 

421 

F-2 


Table  F-3 


ASVAB  Z-Score  Inter correlations  (Mean  =0,  SD  =  1)  for 
Optimal  Test  Lengths  and  180  Minutes  Testing  Time 


AR:  WK:  PC: 

NO: 

CS: 

AS:  MR: 

MC: 

El: 

MEAN: 

SD: 

6S: 

0.610  0.753  0.667 

0.290 

0.262 

0.541  0.561 

0.676 

0.719 

0.000 

1.000 

hRi 

0.591  0.597 

0.471 

0.395 

0.395  0.679 

0.617 

0.531 

0.000 

1 . 000 

HR: 

0.798 

0.340 

0.344 

0.451  0.500 

0.577 

0.611 

0.000 

1.000 

PC: 

0.436 

0.424 

0.367  0.528 

0.537 

0.534 

0.000 

1.000 

NO: 

0.677 

0.049  0.504 

0.242 

0.168 

0.000 

1.000 

CS: 

0.061  0.414 

0.235 

0.170 

0.000 

1.000 

AC  • 

0.197 

0.648 

0.761 

0.000 

1.000 

0.504 

0.410 

0.000 

1.000 

• 

lurr*  • 

0.732 

0.000 

1.000 

El: 

0.000 

1.000 

Test: 

Time  (minutes) : 

Reliability: 

6S: 

16 

.88 

AR: 

26 

.80 

HR: 

16 

.91 

PC: 

21 

.77 

NO: 

5 

.91 

CS: 

10 

.87 

AS: 

13 

.85 

MR: 

20 

.85 

MC: 

26 

.81 

El: 

27 

.87 

TOTAL: 

180 
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Table  F-4 


ASVAB  Validities,  Corrected  for  Criterion  Unreliability  and 
Range  Restriction,  together  with  Sample  Sizes,  in  Eighteen  Joint 
Service  Schools,  with  Optimal  Test  Lengths  and  180  Minutes 

Testing  Time 


6S:  AR:  WK:  PC:  NO: 

AC:  0.646  0.709  0.537  0.616  0.473 
AE:  0.601  0.563  0.555  0.500  0.328 
AMS:  0.726  0.654  0.709  0.643  0.416 
AO:  0.561  0.572  0.513  0.542  0.417 
AV:  0.627  0.713  0.619  0.618  0.410 
EM:  0.552  0.601  0.497  0.483  0.371 
EN:  0.643  0.615  0.590  0.583  0.336 
FC:  0.626  0.667  0.605  0.630  0.417 
GMG:  0.579  0.660  0.609  0.572  0.378 
IIH-H:  0.254  0.231  0.256  0.286  0.088 
IIH-I:  0.329  0.281  0.301  0.312  0.310 
BT/MM:  0.467  0.428  0.366  0.408  0.251 
OS:  0.605  0.683  0.593  0.638  0.499 
RM:  0.635  0.637  0.602  0.575  0.415 
13F:  0.598  0.658  0.589  0.640  0.438 

19K:  0.196  0.181  0.138  0.171  0.187 

272-1:  0.587  0.626  0.653  0.634  0.355 
732:  0.684  0.732  0.671  0.698  0.480 


CS: 

AS: 

MK: 

MC: 

El: 

N: 

0.516 

0.350 

0.770 

0.517 

0.571 

72 

0.366 

0.456 

0.557 

0.566 

0.602 

173 

0.446 

0.538 

0.608 

0.674 

0.674 

244 

0.422 

0.363 

0.610 

0.481 

0.521 

233 

0.382 

0.426 

0.676 

0.544 

0.667 

197 

0.360 

0.357 

0.576 

0.536 

0.529 

805 

0.312 

0.583 

0.545 

0.655 

0.690 

781 

0.398 

0.467 

0.649 

0.648 

0.658 

727 

0.341 

0.456 

0.606 

0.583 

0.585 

393 

0.099 

0.295 

0.246 

0.327 

0.318 

554 

0.231 

0.248 

0.267 

0.359 

0.325 

320 

0.250 

0.411 

0.405 

0.474 

0.476 

837 

0.531 

0.368 

0.686 

0.599 

0.539 

622 

0.431 

0.316 

0.565 

0.548 

0.516 

250 

0.478 

0.463 

0.602 

0.628 

0.559 

819 

0.173 

0.124 

0.199 

0.177 

0.141 

1106 

0.338 

0.441 

0.533 

0.593 

0.573 

484 

0.438 

0.415 

0.681 

0.601 

0.557 

421 

F-4 


Table  F-5 


Testing  Condition  A^:  (1.66) 

DATA  PILE:  PP-IOOP.DAT 

PfiP  100  MINUTE  OPTIMAL  TEST  LENGTH  INTERCORRELATIONS 

AR:  WK:  PC:  NO:  CS:  AS:  MK:  MC:  El:  MEAN:  SD: 


GS:  0.551  0.708 

AR:  0.547 

WK: 

PC: 

NO: 

CS: 

AS: 

MK: 

MC: 

El: 


0.483  0.270  0.244 
0.426  0.432  0.363 
0.592  0.324  0.328 
0.320  0.312 
0.640 


0.496  0.525  0.587 
0.356  0.625  0.527 
0.423  0.478  0.513 
0.266  0.389  0.368 
0.045  0.478  0.213 
0.056  0.393  0.207 
0.183  0.561 
0.445 


0.651  0.000 

0.474  0.000 

0.567  0.000 

0.381  0.000 

0.154  0.000 

0.156  0.000 

0.688  0.000 

0.377  0.000 

0.626  0.000 

0.000 


1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 
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Table  F-6 


Testing  Condition  A^:  (1.66) 

DATA  FILE:  PP~100C.DAT 

P&P  100  MINUTE  OPTIMAL  TEST  LENGTH  VALIDITIES 

6S:  AR:  WK:  PC:  NO:  CS:  AS:  MK:  MC:  El: 

AC:  0.619  0.669  0.527  0.466  0.460  0.502  0.335  0.751  0.469  0.539 

AE:  0.576  0.532  0.545  0.378  0.318  0.356  0.436  0.544  0.512  0.569 

AMS:  0.695  0.617  0.696  0.486  0.404  0.434  0.515  0.593  0.611  0.637 

AO:  0.537  0.540  0.503  0.410  0.405  0.410  0.347  0.595  0.436  0.492 

AV:  0.600  0.673  0.607  0.468  0.398  0.372  0.408  0.659  0.492  0.630 

EM:  0.529  0.567  0.488  0.366  0.360  0.350  0.341  0.562  0.485  0.500 

EN:  0.616  0.580  0.579  0.441  0.326  0.303  0.558  0.531  0.593  0.652 

FC:  0.600  0.629  0.594  0.477  0.405  0.387  0.447  0.633  0.587  0.621 

GMG:  0.555  0.623  0.598  0.433  0.367  0.332  0.436  0.591  0.528  0.553 

IIH-H:  0.243  0.218  0.251  0.216  0.085  0.096  0.282  0.240  0.296  0.300 

0.315  0.266  0.295  0.236  0.302  0.225  0.237  0.260  0.325  0.307 
BT/MM:  0.448  0.404  0.359  0.309  0.244  0.243  0.393  0.395  0.429  0.449 

OS:  0.580  0.644  0.582  0.483  0.485  0.517  0.352  0.669  0.543  0.509 

RM:  0.609  0.601  0.591  0.435  0.404  0.419  0.302  0.551  0.496  0.487 

13F:  0.573  0.621  0.578  0.484  0.426  0.465  0.443  0.588  0.569  0.528 

19K:  0.187  0.171  0.135  0.129  0.182  0.168  0.119  0.194  0.160  0.133 

272-1:  0.562  0.591  0.640  0.480  0.345  0.329  0.422  0.520  0.537  0.542 

732:  0.655  0.691  0.658  0.528  0.467  0.426  0.397  0.664  0.544  0.526 


N: 

72 

173 

244 

233 

197 

805 

781 

727 

393 

554 

320 

837 

622 

250 

819 

1106 

484 

421 


F-6 


Table  F-7 


Testing  Condition  AgS  B2  (1.66) 


DATA  FILE:  B2-100P2.DAT 

CAT-ASVAB  FINAL  HORST  100  MIN  INTERCORRELATIONS 

AR:  WK:  PC:  NO:  CS:  AS:  MK:  MC:  El: 


6S:  0.654 

AR: 

WK: 

PC: 

NO: 

CS: 

AS: 

MK: 

MC: 

El: 


0.793  0.734 
0.633  0.668 
0.877 


0.291  0.264 
0.479  0.404 
0.340  0.345 
0.455  0.445 
0.646 


0.577  0.600 
0.428  0.737 
0.481  0.533 
0.409  0.588 
0.050  0.511 
0.062  0.422 
0.212 


0.733  0.767 
0.679  0.576 
0.624  0.651 
0.606  0.594 
0.249  0.170 
0.243  0.173 
0.710  0.822 
0.553  0.442 
0.801 


MEAN: 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 


SD: 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 
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Table  F-8 


Testing  Condition  Agt  (1.66) 

DATA  FILE:  B2-100C2.DAT 

CAT-ASVAB  FINAL  HORST  100  MIN  VALIDITIES 


GS: 

AR: 

WK: 

PC: 

NO: 

CS: 

AS: 

MK: 

MC: 

£1: 

N: 

AC 

0.664 

0.741 

0.551 

0.660 

0.461 

0.505 

0.363 

0.801 

0.545 

0.593 

72 

A£ 

0.618 

0 . 588 

0.570 

0.536 

0.319 

0.359 

0.473 

0.579 

0.596 

0.625 

173 

AMS 

0.745 

0.683 

0.727 

0.689 

0.405 

0.437 

0.559 

0.632 

0.711 

0.700 

244 

AOs 

0.576 

0.597 

0.526 

0.581 

0.406 

0.413 

0.377 

0.634 

0.508 

0.541 

233 

AVj 

0.644 

0.745 

0.635 

0.663 

0.399 

0.374 

0.443 

0.703 

0.573 

0.692 

197 

£M ! 

0.567 

0.627 

0.510 

0.518 

0.361 

0.352 

0.371 

0.599 

0.565 

0.549 

805 

EN: 

0.661 

0.642 

0.605 

0.625 

0.327 

0.305 

0.606 

0.566 

0.691 

0.717 

781 

FCs 

0.643 

0.697 

0.621 

0.675 

0.406 

0.390 

0.486 

0.675 

0.683 

0.683 

727 

6MG: 

0 . 595 

0.689 

0.625 

0.613 

0.368 

0.334 

0.474 

0.630 

0.615 

0.608 

393 

0.242 

0.262 

0.307 

0.085 

0.097 

0.306 

0.256 

0.345 

0.330 

554 

liH-J 

::  0.338 

0.294 

0.309 

0.334 

0.302 

0.227 

0.257 

0.277 

0.378 

0.337 

320 

hx/mm:  0.480 

0.448 

0.375 

0.437 

0.244 

0.245 

0.427 

0.421 

0.499 

0.494 

837 

os: 

0.621 

0.713 

0.608 

0.684 

0.486 

0.520 

0.383 

0.713 

0.632 

0.559 

622 

RM: 

0.653 

0.665 

0.618 

0.616 

0.405 

0.422 

0.328 

0.587 

0.578 

0.536 

250 

13F: 

0.614 

0.688 

0.604 

0.685 

0.427 

0.468 

0.481 

0.627 

0.663 

0.580 

819 

19K: 

0.201 

0.189 

0.142 

0.183 

0.182 

0.170 

0.129 

0.207 

0.186 

0.146 

1106 

272-1 

:  0.603 

0.654 

0.669 

0.679 

0.346 

0.331 

0.458 

0.554 

0.625 

0.595 

484 

732 : 

0.702 

0.765 

0.688 

0.748 

0.468 

0.429 

0.431 

0.708 

0.633 

0.578 

421 

P-8 


Table  F-9 


Testing  Condition  A^:  (2.25) 

DATA  FILE:  B3-135P1.DAT 

P&P  ASVAB  +  ECAT  FINAL  HORST  135  MIN  INTERCORRELATIONS 


AR:  WK:  PC:  NO: 

GS:  0.566  0.723  0.555  0.279 

AR:  0.554  0.485  0.442 

WK:  0.669  0.330 

PC:  0.366 

NO: 

CS: 

AS: 

MK: 

HC: 

El: 


CS:  AS:  MK:  MC:  El: 

0.251  0.513  0.537  0.612  0.677 
0.370  0.366  0.634  0.545  0.488 
0.332  0.432  0.483  0.527  0.580 
0.355  0.305  0.441  0.425  0.439 
0.656  0.047  0.488  0.222  0.160 
0.058  0.399  0.214  0.161 
0.188  0.585  0.714 
0.459  0.388 
0.655 


FR: 

AO: 

SO: 

MEAN; 

SD: 

6S: 

0.464 

0.360 

0.481 

0.000 

1.000 

AR: 

0.512 

0.361 

0.490 

0.000 

1.000 

WK: 

0.441 

0.304 

0.434 

0.000 

1.000 

PC: 

0.374 

0.248 

0.352 

0.000 

1.000 

NO: 

0.290 

0.183 

0.225 

0.000 

1.000 

CS: 

0.270 

0.203 

0.238 

0.000 

1.000 

AS: 

0.283 

0.292 

0.380 

0.000 

1.000 

MK: 

0.490 

0.343 

0.460 

0.000 

1.000 

MC: 

0.473 

0.407 

0.529 

0.000 

1.000 

El: 

0.388 

0.351 

0.458 

0.000 

1.000 

FR: 

0.408 

0.498 

0.000 

1.000 

AO: 

0.437 

0.000 

1.000 

SO: 

0.000 

1.000 
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Table  F-10 


Testing  Condition  A^:  Bj 

DATA  PILE:  B3-135C1.DAT 

P&P  ASVAB  +  ECAT  FINAL  HORST  135  MIN  VALIDITIES 


GS: 

AR: 

WK: 

PC: 

NO: 

CS: 

AS: 

AC 

0.631 

0.676 

0.528 

0.525 

0.466 

0.507 

0.340 

A£ 

0.587 

0.537 

0.547 

0.426 

0.323 

0.360 

0.443 

AMS 

0.708 

0.623 

0.698 

0.548 

0.410 

0.438 

0.523 

AO 

0.547 

0.545 

0.504 

0.462 

0.411 

0.414 

0.353 

AVj 

0.611 

0.680 

0.609 

0.527 

0.404 

0.375 

0.415 

EM: 

0.538 

0.572 

0.489 

0.412 

0.365 

0.353 

0.347 

EN: 

0.627 

0.586 

0.580 

0.497 

0.331 

0.306 

0.568 

FCi 

0.611 

0.635 

0.595 

0.537 

0.411 

0.391 

0.454 

GM6: 

0.565 

0.629 

0.599 

0.488 

0.373 

0.335 

0.443 

IIH-H:  0.248 

0.220 

0.252 

0.244 

0.087 

0.097 

0.287 

llH-1:  0.321 

0.268 

0.296 

0.266 

0.306 

0.227 

0.241 

BT/MM:  0.456 

0.408 

0.360 

0.348 

0.247 

0.245 

0.400 

OSi 

0.590 

0.651 

0.584 

0.544 

0.492 

0.522 

0.358 

RM: 

0.620 

0.607 

0.593 

0.490 

0.409 

0.423 

0.307 

13F: 

0.584 

0.627 

0.580 

0.545 

0.432 

0.470 

0.451 

19K: 

0.191 

0.173 

0.136 

0.146 

0.184 

0.170 

0.121 

272-3 

.:  0.573 

0.597 

0.642 

0.540 

0.350 

0.332 

0.429 

732: 

0.667 

0.698 

0.660 

0.595 

0.474 

0.430 

0.404 

FR: 

AO: 

SO: 

N; 

AC: 

0.545 

0.404 

0.474 

72 

AE: 

0.469 

0.411 

0.508 

173 

AMS: 

0.482 

0.383 

0.486 

244 

AO: 

0.439 

0.367 

0.440 

233 

AV: 

0.496 

0.385 

0.472 

197 

EM: 

0.437 

0.316 

0.432 

805 

EN: 

0.455 

0.365 

0.494 

781 

FC: 

0.518 

0.399 

0.523 

727 

6MG: 

0.493 

0.356 

0.484 

393 

0.205 

0.228 

0.298 

554 

IIH-I: 

0.203 

0.220 

0.320 

320 

BT/MM: 

0.383 

0.320 

0.321 

837 

OS: 

0.525 

0.388 

0.517 

622 

RM: 

0.440 

0.344 

0.466 

250 

13F: 

0.547 

0.434 

0.563 

819 

19K: 

0.105 

0.130 

0.134 

1106 

272-1: 

0.462 

0.377 

0.530 

484 

732: 

0.566 

0.378 

0.544 

421 

(2.25) 


MK:  MC: 

0.756  0.480 
0.547  0.525 
0.597  0.626 
0.598  0.447 
0.663  0.504 
0.565  0.497 
0.534  0.608 
0.637  0.601 
0.594  0.541 
0.241  0.304 
0.261  0.333 
0.397  0.440 
0.673  0.556 
0.554  0.509 
0.591  0.583 
0.195  0.164 
0.522  0.550 
0.668  0.558 


El:  N: 

0.551  72 

0.581  173 

0.650  244 

0.502  233 

0.643  197 

0.510  805 

0.666  781 

0.635  727 

0.565  393 

0.307  554 

0.313  320 

0.459  837 

0.519  622 

0.497  250 

0.539  819 

0.136  1106 
0.553  484 

0.537  421 


P-10 


Table  F-11 


Testing  Condition  A^;  (2.25) 

DATA  FILE:  B4-135P3.DAT 

CAT-ASVAB+NON-PEDESTAL  ECAT  FOURTH  HORST  135  MIN  INTERCORRELATIONS 
AR;  WK:  PC;  NO:  CS:  AS:  MK:  MC:  El: 


GS:  0.658 
AR: 

WK: 

PC: 

NO: 

CS: 

AS: 

MK: 

MC: 

El: 


0.797  0.749 
0.634  0.680 
0.891 


0.294  0.259 
0.483  0.396 
0.342  0.337 
0.465  0.442 
0.636 


0.584  0.603 
0.432  0.740 
0.484  0.534 
0.419  0.599 
0.050  0.515 
0.061  0.413 
0.214 


0.741  0.774 
0.685  0.580 
0.629  0.654 
0.621  0.606 
0.253  0.172 
0.239  0.169 
0.720  0.831 
0.558  0.445 
0.811 


CT:  SM:  FR:  ID:  AO:  SO;  MEAN:  SD; 


GS: 

0.349 

0.329 

0.492 

0.229 

0.413 

0.494 

AR: 

0.505 

0.459 

0.563 

0.249 

0.430 

0.523 

WK: 

0.319 

0.328 

0.461 

0.193 

0.344 

0.439 

PC: 

0.363 

0.368 

0.479 

0.194 

0.344 

0.437 

NO: 

0.333 

0.299 

0.290 

0.111 

0.197 

0.218 

CS: 

0.305 

0.288 

0.264 

0.108 

0.215 

0.225 

AS: 

0.195 

0.152 

0.306 

0.173 

0.341 

0.398 

MK: 

0.473 

0.428 

0.521 

0.228 

0.396 

0.475 

MC: 

0.418 

0.356 

0.543 

0.271 

0.506 

0.588 

El: 

0.281 

0.238 

0.420 

0.216 

0.412 

0.482 

CT: 

SM: 

FR; 

ID: 

AO: 

SO: 

0.472 

0.464 

0.429 

0.212 

0.181 

0.239 

0.422 

0.345 

0.444 

0.231 

0.431 

0.372 

0.485 

0.237 

0.461 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 


1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


Table  F-12 


Testing  Condition  A^:  (2.25) 


DATA  FILE:  B4-135C3.DAT 

CAT-ASVAB+NON-PEDESTAL  ECAT  FOURTH  HORST  135  MIN  VALIDITIES 


GS:  AR: 


AC 

0.667 

0.742 

AE 

0.620 

0.589 

AMS 

0.748 

0.684 

AO] 

0.578 

0.598 

AVj 

0.646 

0.746 

EM; 

0.569 

0.628 

EN; 

0.663 

0.643 

FC; 

0.646 

0.698 

GMG: 

0.597 

0.690 

IIH-H:  0.262 

0.242 

IIH-I:  0.339 

0.294 

BT/MM:  0.482 

0.448 

OS: 

0.624 

0.714 

RM: 

0.655 

0.666 

13F: 

0.617 

0.689 

19K: 

0.202 

0.190 

272-1 

.:  0.605 

0.655 

732: 

0.705 

0.766 

CT; 

SM: 

AC: 

0.481 

0.423 

AE: 

0.458 

0.396 

AMS: 

0.375 

0.358 

AO: 

0.367 

0.293 

AV: 

0.463 

0.408 

EM: 

0.399 

0.315 

EN: 

0.365 

0.296 

FC: 

0.407 

0.362 

GMG: 

0.441 

0.340 

IIH-H: 

0.207 

0.171 

IIH-I: 

0.236 

0.262 

BT/MM: 

0.283 

0.235 

OS: 

0.502 

0.432 

RM: 

0.439 

0.410 

13F: 

0.482 

0.440 

19K: 

0.141 

0.131 

272-1: 

0.456 

0.384 

732: 

0.453 

0.463 

WK:  PC:  NO: 

0.551  0.671  0.464 
0.570  0.544  0.321 
0.727  0.700  0.408 
0.526  0.590  0.409 
0.635  0.673  0.402 
0.510  0.526  0.364 
0.605  0.635  0.329 
0.621  0.686  0.409 
0.625  0.623  0.371 
0.262  0.312  0.086 
0.309  0.339  0.304 
0.375  0.444  0.246 
0.608  0.695  0.490 
0.618  0.626  0.407 
0.604  0.697  0.430 
0.142  0.186  0.183 
0.669  0.690  0.348 
0.688  0.760  0.471 


FR:  ID:  AO: 

0.547  0.207  0.438 
0.470  0.246  0.446 
0.483  0.234  0.415 
0.440  0.201  0.398 
0.498  0.232  0.418 
0.438  0.200  0.343 
0.456  0.219  0.396 
0.519  0.238  0.433 
0.495  0.215  0.386 
0.205  0.109  0.247 
0.203  0.135  0.239 
0.384  0.167  0.348 
0.527  0.233  0.421 
0.441  0.221  0.373 
0.549  0.255  0.471 
0.105  0.068  0.141 
0.463  0.211  0.409 
0.568  0.238  0.410 


CS:  AS;  MK: 

0.494  0.366  0.803 
0.350  0.477  0.581 
0.427  0.563  0.634 
0.404  0.380  0.636 
0.366  0.446  0.704 
0.344  0.373  0.601 
0.298  0.611  0.568 
0.381  0.489  0.676 
0.327  0.477  0.631 
0.094  0.309  0.256 
0.221  0.259  0.278 
0.239  0.430  0.422 
0.508  0.386  0.715 
0.413  0.331  0.589 
0.457  0.485  0.628 
0.166  0.130  0.207 
0.324  0.461  0.555 
0.419  0.435  0.710 


SO:  N; 

0.461  72 

0.493  173 

0.472  244 

0.427  233 

0.459  197 

0.420  805 

0.480  781 

0.508  727 

0.470  393 

0.289  554 

0.310  320 

0.312  837 

0.502  622 

0.453  250 

0.547  819 

0.130  1106 
0.515  484 

0.528  421 


MC: 

El: 

N: 

0.550 

0.595 

72 

0.601 

0.628 

173 

0.716 

0.703 

244 

0.512 

0.543 

233 

0.578 

0.695 

197 

0.569 

0.552 

805 

0.696 

0.720 

781 

0.688 

0.686 

727 

0.620 

0.610 

393 

0.347 

0.332 

554 

0.381 

0.339 

320 

0.503 

0.496 

837 

0.636 

0.562 

622 

0.582 

0.538 

250 

0.668 

0.583 

819 

0.188 

0.147 

1106 

0.630 

0.598 

484 

0.638 

0.580 

421 

Tfible  F-13 


Testing  Condition  A^:  (3.00) 

DATA  FILE:  PP-180P.DAT 

P&P  180  MINUTE  OPTIMAL  TEST  LENGTH  INTERCOBRELATIONS 


AR:  WK: 

PC: 

NO: 

CS: 

AS: 

MK: 

MC: 

El: 

MEAN: 

SD: 

6S: 

0.610  0.753 

0.667 

0.290 

0.262 

0.541 

0.561 

0.676 

0.719 

0.000 

1.000 

AR: 

0.591 

0.597 

0.471 

0.395 

0.395 

0.679 

0.617 

0.531 

0.000 

1.000 

WK: 

0.798 

0.340 

0.344 

0.451 

0.500 

0.577 

0.611 

0.000 

1.000 

PC; 

0.436 

0.424 

0.367 

0.528 

0.537 

0.534 

0.000 

1.000 

NO: 

0.677 

0.049 

0.504 

0.242 

0.168 

0.000 

1.000 

CS: 

0.061 

0.414 

0.235 

0.170 

0.000 

1.000 

AS: 

0.197 

0.648 

0.761 

0.000 

1.000 

MK: 

0.504 

0.410 

0.000 

1.000 

MC: 

0.732 

0.000 

1.000 

El: 

0.000 

1.000 

F-13 


Table  F-14 


Testing  Condition  A^:  (3.00) 

DATA  FILE:  PP-180C.DAT 

P&P  180  MINUTE  OPTIMAL  TEST  LENGTH  VALIDITIES 


GS:  AR:  WK:  PC: 


NO:  CS: 


AS:  MK:  MC: 


El: 


AC: 
AE: 
AMS: 
AO: 
AV: 
EM: 
EN: 
FC: 
GMG: 
IIH-H: 
IIH-I : 
BT/MM: 
OS: 
RM: 
13F: 
19K: 
272-1: 
732: 


0.646  0.709 
0.601  0.563 
0.726  0.654 
0.561  0.572 
0.627  0.713 
0.552  0.601 
0.643  0.615 
0.626  0.667 
0.579  0.660 
0.254  0.231 
0.329  0.281 
0.467  0.428 
0.605  0.683 
0.635  0.637 
0.598  0.658 
0.196  0.181 
0.587  0.626 
0.684  0.732 


0.537 

0.616 

0.555 

0.500 

0.709 

0.643 

0.513 

0.542 

0.619 

0.618 

0.497 

0.483 

0.590 

0.583 

0.605 

0.630 

0.609 

0.572 

0.256 

0.286 

0.301 

0.312 

0.366 

0.408 

0.593 

0.638 

0.602 

0.575 

0.589 

0.640 

0.138 

0.171 

0.653 

0.634 

0.671 

0.698 

0.473  0.516 
0.328  0.366 
0.416  0.446 
0.417  0.422 
0.410  0.382 
0.371  0.360 
0.336  0.312 
0.417  0.398 
0.378  0.341 
0.088  0.099 
0.310  0.231 
0.251  0.250 
0.499  0.531 
0.415  0.431 
0.438  0.478 
0.187  0.173 
0.355  0.338 
0.480  0.438 


0.350  0.770 
0.456  0.557 
0.538  0.608 
0.363  0.610 
0.426  0.676 
0.357  0.576 
0.583  0.545 
0.467  0.649 
0.456  0.606 
0.295  0.246 
0.248  0.267 
0.411  0.405 
0.368  0.686 
0.316  0.565 
0.463  0.602 
0.124  0.199 
0.441  0.533 
0.415  0.681 


0.517 

0.571 

0.566 

0.602 

0.674 

0.674 

0.481 

0.521 

0.544 

0.667 

0.536 

0.529 

0.655 

0.690 

0.648 

0.658 

0.583 

0.585 

0.327 

0.318 

0.359 

0.325 

0.474 

0.476 

0.599 

0.539 

0.548 

0.516 

0.628 

0.559 

0.177 

0.141 

0.593 

0.573 

0.601 

0.557 

N: 

72 

173 

244 

233 

197 

805 

781 

727 

393 

554 

320 

837 

622 

250 

819 

1106 

484 

421 
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Table  F-15 


Testing  Condition  A^:  (3.00) 


DATA  FILE:  B3-180P1.DAT 

P&P  ASVAB  ECAT  FINAL  HORST  180  MIN  INTERCORRELATIONS 


AR: 

6S:  0.595 


WK:  PC: 

0.745  0.642 
0.578  0.569 
0.770 


NO:  CS: 

0.287  0.259 
0.461  0.386 
0.337  0.340 
.421  0.409 
0.671 


AS:  MK: 

0.532  0.554 
0.384  0.663 
0.445  0.495 
0.353  0.508 
0.048  0.499 
0.060  0.409 
0.194 


MC:  El: 

0.659  0.708 
0.594  0.517 
0.563  0.603 
0.510  0.513 
0.237  0.165 
0.229  0.167 
0.629  0.748 
0.491  0.403 
0.711 


FR: 

AO: 

SO: 

MEAN: 

SD: 

GS: 

0.487 

0.399 

0.499 

0.000 

1.000 

AR: 

0.544 

0.405 

0.516 

0.000 

1.000 

WK: 

0.460 

0.335 

0.447 

0.000 

1.000 

PC: 

0.438 

0.307 

0.408 

0.000 

1.000 

NO: 

0.302 

0.201 

0.232 

0.000 

1.000 

CS: 

0.282 

0.224 

0.245 

0.000 

1.000 

AS: 

0.297 

0.324 

0.395 

0.000 

1.000 

MK: 

0.511 

0.378 

0.475 

0.000 

1.000 

MC: 

0.515 

0.469 

0.569 

0.000 

1.000 

EX: 

0.410 

0.393 

0.480 

0.000 

1.000 

FR: 

0.457 

0.523 

0.000 

1.000 

AO: 

0.485 

0.000 

1.000 

SO: 

0.000 

1.000 
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Table  P-16 


Testing  Condition  A^:  (3.00) 

DATA  FILE;  B3-180C1.DAT 

P&P  +  ECAT  FINAL  HORST  180  MIN  VALIDITIES 


GS: 

AR: 

WK: 

PC: 

NO: 

CS: 

AS: 

HK: 

HC: 

El: 

N: 

AC: 

0.642 

0.697 

0.534 

0.597 

0.471 

0.513 

0.347 

0.765 

0.507 

0.566 

72 

AE: 

0.597 

0.554 

0.553 

0.485 

0.326 

0.364 

0.452 

0.554 

0.555 

0.597 

173 

AMS: 

0.721 

0.642 

0.706 

0.623 

0.414 

0.444 

0.533 

0.604 

0.661 

0.668 

244 

AO: 

0.557 

0.562 

0.510 

0.526 

0.415 

0.419 

0.359 

0.606 

0.472 

0.516 

233 

AV: 

0.623 

0.701 

0.617 

0.600 

0.408 

0.380 

0.422 

0.671 

0.533 

0.660 

197 

EM: 

0.548 

0.590 

0.495 

0.469 

0.369 

0.358 

0.354 

0.573 

0.525 

0.524 

805 

EN: 

0.639 

0.604 

0.587 

0.565 

0.334 

0.310 

0.578 

0.541 

0.643 

0.684 

781 

FC: 

0.622 

0.655 

0.603 

0.611 

0.416 

0.396 

0.463 

0.645 

0.635 

0.652 

727 

6MG: 

0.575 

0.648 

0.607 

0.555 

0.376 

0.339 

0.452 

0.602 

0.572 

0.580 

393 

IIH-H: 

0.252 

0.227 

0.255 

0.277 

0.087 

0.098 

0.292 

0.244 

0.321 

0.315 

554 

IIH-I: 

0.327 

0.277 

0.300 

0.302 

0.309 

0.230 

0.246 

0.265 

0.352 

0.322 

320 

BT/MM: 

0.464 

0.421 

0.364 

0.396 

0.250 

0.248 

0.407 

0.402 

0.464 

0.471 

837 

OS: 

0.601 

0.671 

0.591 

0.619 

0.497 

0.528 

0.365 

0.681 

0.587 

0.534 

622 

RM: 

0.631 

0.626 

0.600 

0.557 

0.414 

0.429 

0.313 

0.561 

0.537 

0.511 

250 

13F: 

0.594 

0.647 

0.587 

0.620 

0.436 

0.475 

0.459 

0.599 

0.616 

0.553 

819 

19K: 

0.194 

0.178 

0.138 

0.166 

0.186 

0.172 

0.123 

0.198 

0.173 

0.139 

1106 

272-1: 

0.583 

0.616 

0.650 

0.615 

0.354 

0.336 

0.437 

0.529 

0.581 

0.568 

484 

732: 

0.679 

0.719 

0.668 

0.676 

0.478 

0.436 

0.412 

0.677 

0.589 

0.551 

421 

FR; 

AO: 

SO: 

N; 

AC: 

0.562 

0.439 

0.483 

72 

AE: 

0.483 

0.448 

0.518 

173 

AMS: 

0.496 

0.417 

0.496 

244 

AO: 

0.452 

0.399 

0.448 

233 

AV: 

0.512 

0.419 

0.481 

197 

EM: 

0.450 

0.344 

0.440 

805 

EN: 

0.469 

0.397 

0.504 

781 

FC: 

0.533 

0.434 

0.533 

727 

GM6: 

0.508 

0.387 

0.493 

393 

IIH-H: 

0.211 

0.248 

0.304 

554 

IIH-I: 

0.209 

0.240 

0.326 

320 

BT/MM: 

0.395 

0.349 

0.328 

837 

OS: 

0.541 

0.422 

0.527 

622 

RM: 

0.453 

0.374 

0.475 

250 

13F: 

0.564 

0.472 

0.574 

819 

19K: 

0.108 

0.141 

0.136 

1106 

272-1: 

0.476 

0.410 

0.540 

484 

732: 

0.583 

0.411 

0.554 

421 
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Table  F-17 


Testing  Condition  A^:  (3.00) 

DATA  FILE:  B4-180P1.DAT 

CAT-ASVAB  +  NON-PEDESTAL  ECAT  SECOND  HORST  180  MIN  INTERCORRELATIONS 
AR;  WK:  PC:  NO:  CS:  AS:  MK:  MC:  El: 

GS: 

AR: 

WK: 

PC: 

NO: 

CS: 

AS: 

MK: 

MC: 

El: 


0.668  0.806  0.764  0.300  0.268  0.590  0.609  0.756  0.783 

0.646  0.698  0.497  0.412  0.440  0.752  0.704  0.590 

0.911  0.351  0.350  0.491  0.541  0.643  0.663 
0.481  0.462  0.428  0.611  0.640  0.619 

0.666  0.051  0.527  0.261  0.176 

0.063  0.427  0.250  0.175 
0.217  0.736  0.842 
0.570  0.451 
0.829 


CT:  SM:  FR:  ID: 

GS:  0.369  0.358  0.525  0.375 

AR:  0.539  0.504  0.605  0.411 

WK:  0.338  0.358  0.493  0.318 

PC:  0.388  0.405  0.516  0.321 

NO:  0.358  0.330  0.313  0.184 

CS:  0.331  0.322  0.289  0.181 

AS:  0.207  0.166  0.326  0.284 

MK:  0.502  0.467  0.556  0.374 

MC:  0.448  0.392  0.586  0.449 

El:  0.298  0.259  0.449  0.354 

CT:  0.539  0.520  0.364 

SM:  0.493  0.321 

FR:  0.413 

ID: 

AO: 

SO: 


AOs 

SO: 

MEAN: 

SD: 

0.458 

0.518 

0.000 

1.000 

0.480 

0.553 

0.000 

1.000 

0.382 

0.462 

0.000 

1.000 

0.385 

0.463 

0.000 

1.000 

0.221 

0.232 

0.000 

1.000 

0.244 

0.242 

0.000 

1.000 

0.379 

0.418 

0.000 

1.000 

0.439 

0.498 

0.000 

1.000 

0.567 

0.623 

0.000 

1.000 

0.458 

0.506 

0.000 

1.000 

0.491 

0.474 

0.000 

1.000 

0.413 

0.421 

0.000 

1.000 

0.520 

0.538 

0.000 

1.000 

0.416 

0.404 

0.000 

1.000 

0.530 

0.000 

1.000 

0.000 

1.000 
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Table  F-18 


Testing  Condition  A^:  (3.00) 

DATA  FILE:  B4-180C1.DAT 

CAT-ASVAB  +  NON-PEDESTAL  ECAT  SECOND  HORST  180  MIN  VALIDITIES 


GS:  AR:  WK:  PC:  NO:  CS: 

AC:  0.670  0.751  0.554  0.681  0.472  0.508 

AE:  0.623  0.596  0.574  0.553  0.327  0.361 

AMS:  0.752  0.692  0.732  0.710  0.415  0.439 

AO:  0.581  0.605  0.530  0.599  0.416  0.415 

AV:  0.650  0.755  0.640  0.684  0.409  0.377 

EM:  0.572  0.636  0.514  0.534  0.370  0.354 

EN:  0.667  0.650  0.609  0.645  0.335  0.307 

FC:  0.649  0.706  0.625  0.696  0.417  0.392 

GMG:  0.600  0.698  0.629  0.633  0.377  0.336 

IIH-H:  0.263  0.245  0.264  0.316  0.088  0.097 

IIH-I:  0.341  0.298  0.311  0.344  0.310  0.228 

BT/MM:  0.484  0.453  0.378  0.451  0.250  0.246 

OS:  0.627  0.723  0.613  0.705  0.498  0.523 

RM:  0.658  0.674  0.622  0.635  0.415  0.425 

13F:  0.620  0.697  0.609  0.707  0.437  0.471 

19K:  0.203  0.192  0.143  0.189  0.186  0.170 

272-1:  0.608  0.663  0.674  0.701  0.354  0.333 

732:  0.708  0.775  0.693  0.771  0.479  0.432 


AS: 

MK: 

MC; 

El: 

N: 

0.368 

0.807 

0.558 

0.599 

72 

0.480 

0.584 

0.610 

0.632 

173 

0.566 

0.637 

0.727 

0.708 

244 

0.382 

0.639 

0.519 

0.547 

233 

0.449 

0.708 

0.586 

0.700 

197 

0.376 

0.604 

0.578 

0.556 

805 

0.614 

0.570 

0.707 

0.725 

781 

0.492 

0.680 

0.699 

0.691 

727 

0.480 

0.634 

0.629 

0.615 

393 

0.310 

0.257 

0.353 

0.334 

554 

0.261 

0.279 

0.387 

0.341 

320 

0.433 

0.424 

0.511 

0.500 

837 

0.388 

0.718 

0.646 

0.565 

622 

0.333 

0.591 

0.591 

0.542 

250 

0.488 

0.631 

0.678 

0.587 

819 

0.131 

0.208 

0.191 

0.148 

1106 

0.464 

0.558 

0.639 

0.602 

484 

0.437 

0.713 

0.648 

0.585 

421 

AC: 

AE: 

AMS: 

AO: 

AV: 

EM: 

EN: 

PC: 

GMG: 

IIH-H: 

IIH-I: 

BT/MM: 

OS: 

RM: 

13P: 

19K: 

272-1: 


CT:  SM:  FR:  ID:  AO:  SO:  N; 


0.507 

0.482 

0.395 

0.386 

0.488 

0.420 

0.385 

0.429 

0.465 

0.218 

0.249 

0.298 

0.529 

0.463 

0.508 

0.149 

0.480 


0.459 

0.429 

0.388 

0.318 

0.442 

0.342 

0.321 

0.392 

0.368 

0.185 

0.284 

0.254 

0.468 

0.445 

0.477 

0.141 

0.416 


0.580 

0.499 

0.513 

0.467 

0.529 

0.465 

0.484 

0.551 

0.525 

0.218 

0.216 

0.408 

0.559 

0.468 

0.583 

0.111 

0.492 


0.338 

0.401 

0.381 

0.328 

0.378 

0.327 

0.357 

0.388 

0.350 

0.178 

0.220 

0.272 

0.380 

0.361 

0.417 

0.111 

0.345 


0.483 

0.492 

0.458 

0.439 

0.461 

0.378 

0.436 

0.478 

0.426 

0.273 

0.264 

0.384 

0.464 

0.412 

0.520 

0.155 

0.451 


0.481 

0.515 

0.493 

0.446 

0.479 

0.438 

0.501 

0.530 

0.491 

0.302 

0.324 

0.326 

0.524 

0.473 

0.571 

0.136 

0.537 


72 

173 

244 

233 

197 

805 

781 

727 

393 

554 

320 

837 

622 

250 

819 

1106 

484 


Table  F-19 


Testing  Condition  Agt  Bj  (3.00) 


DATA  FILE:  B5-180P1.DAT 

CAT-ASVAB  +  ALL  BCAT  TESTS  SECOND  HORST  180  MIN  INTERCORRELATIONS 


AR: 

WK: 

PC: 

NO: 

CS: 

AS:  MK:  MC:  El: 

GS: 

0.657 

0.805 

0.760 

0.298 

0.268 

0.588  0.608  0.749  0.781 

AR: 

0.634 

0.684 

0.485 

0.405 

0.431  0.738  0.686  0.579 

WK: 

0.905 

0.348 

0.349 

0.488  0.539  0.636  0.661 

PC: 

0.475 

0.460 

0.424  0.606  0.631  0.615 

NO: 

0.661 

0.051  0.522  0.257  0.174 

CS: 

0.063  0.426  0.247  0.175 

AS: 

0.216  0.726  0.837 

MK: 

0.563  0.448 

MC: 

0.819 

El: 

CT: 

SM: 

FR: 

ID: 

AO: 

SO:  Tl:  T2:  TI: 

MEAN: 

SD: 

GS: 

0.368 

0.352 

0.524 

0.332 

0.417 

0.505-0.318-0.371-0.338 

0.000 

1.000 

AR: 

0.528 

0.486 

0.594 

0.358 

0.430 

0 . 529-0 . 311-0 . 347-0 . 269 

0.000 

1.000 

WK: 

0.337 

0.351 

0.492 

0.281 

0.347 

0 . 449-0 . 270-0 . 323-0 . 269 

0.000 

1.000 

PC: 

0.385 

0.396 

0.513 

0.283 

0.349 

0 . 449-0 . 281-0 . 317-0 . 266 

0.000 

1.000 

NO: 

0.354 

0.322 

0.310 

0.162 

0.200 

0 .224-0 . 222-0 . 203-0 . 189 

0.000 

1.000 

CS: 

0.330 

0.316 

0.288 

0.160 

0.222 

0.236-0.2 14-0 . 222-0 . 202 

0.000 

1.000 

AS: 

0.206 

0.162 

0.324 

0.251 

0.344 

0 . 406-0 . 288-0 . 348-0 . 244 

0.000 

1.000 

MK: 

0.499 

0.457 

0.554 

0.330 

0.398 

0 . 484-0 . 276-0 . 290-0 .236 

0.000 

1.000 

MC: 

0.442 

0.381 

0.579 

0.394 

0.511 

0 . 601-0 . 426-0 . 490-0 . 365 

0.000 

1.000 

EX: 

0.296 

0.254 

0.447 

0.313 

0.416 

0.492-0.325-0.390-0.274 

0.000 

1.000 

CT: 

0.528 

0.517 

0.321 

0.445 

0.460-0.371-0.368-0.282 

0.000 

1.000 

SM: 

0.483 

0.279 

0.369 

0 . 403-0 . 306-0 . 307-0 . 2  58 

0.000 

1.000 

FR: 

0.365 

0.472 

0 . 523-0 . 358-0 . 371-0 . 292 

0.000 

1.000 

ID: 

0.336 

0 . 349-0 . 247-0 . 257-0 . 207 

0.000 

1.000 

AO: 

0 . 471-0 . 334-0 . 362-0 . 309 

0.000 

1.000 

SO: 

-0.372-0.399-0.276 

0.000 

1.000 

Tl: 

0.808  0.396 

0.000 

1.000 

T2: 

0.401 

0.000 

1.000 

TI: 

0.000 

1.000 
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Table  F-20 


Testing  Condition  Agt  Bj  (3.00) 

DATA  FILE:  B5-180C1.DAT 

CAT-ASVAB  +  ALL  ECAT  TESTS  SECOND  HORST  180  MIN  VALIDITIES 

GS:  AR:  WK:  PC:  NO:  CS:  AS:  MK:  MC:  El:  N: 

AC:  0.670  0.738  0.554  0.678  0.469  0.508  0.367  0.805  0.553  0.598  72 

AE:  0.623  0.586  0.573  0.550  0.325  0.361  0.478  0.582  0.604  0.631  173 

AMS:  0.752  0.680  0.731  0.707  0.412  0.439  0.564  0.635  0.720  0.706  244 

AO:  0.581  0.595  0.529  0.596  0.413  0.415  0.380  0.637  0.514  0.545  233 

AV:  0.650  0.742  0.639  0.680  0.406  0.377  0.447  0.706  0.581  0.698  197 

EM:  0.572  0.625  0.513  0.532  0.367  0.354  0.374  0.602  0.572  0.554  805 

EN:  0.667  0.640  0.608  0.642  0.333  0.307  0.612  0.569  0.700  0.723  781 

FC:  0.649  0.694  0.624  0.693  0.414  0.392  0.490  0.678  0.692  0.689  727 

GMG:  0.600  0.687  0.628  0.630  0.374  0.336  0.478  0.633  0.623  0.613  393 

IIH-H:  0.263  0.241  0.264  0.315  0.087  0.097  0.309  0.257  0.349  0.333  554 

IIH-I:  0.341  0.293  0.310  0.343  0.308  0.228  0.260  0.278  0.383  0.340  320 

BT/MM;  0.484  0.446  0.377  0.449  0.249  0.246  0.431  0.423  0.506  0.499  837 

OS:  0.627  0.711  0.612  0.702  0.495  0.523  0.387  0.716  0.640  0.564  622 

RM:  0.658  0.663  0.621  0.632  0.412  0.425  0.331  0.590  0.586  0.540  250 

13F:  0.620  0.685  0.608  0.704  0.434  0.471  0.486  0.629  0.671  0.585  819 

19K:  0.203  0.189  0.142  0.188  0.185  0.171  0.130  0.208  0.189  0.147  1106 

272-1:  0.608  0.652  0.673  0.698  0.352  0.333  0.462  0.556  0.633  0.601  484 

732:  0.708  0.762  0.692  0.768  0.476  0.432  0.436  0.712  0.642  0.583  421 


CT:  SM:  FR:  ID:  AO:  SO:  Tl:  T2:  TI:  N; 

AC:  0.506  0.450  0.579  0.300  0.440  0.468-0.238-0.286-0.240  72 

AE:  0.481  0.421  0.498  0.355  0.448  0.502-0.382-0.353-0.369  173 

AMS:  0.394  0.381  0.512  0.338  0.417  0.480-0.312-0.355-0.295  244 

AO:  0.385  0.312  0.466  0.291  0.400  0.435-0.373-0.323-0.345  233 

AV:  0.486  0.434  0.528  0.335  0.420  0.467-0.273-0.378-0.273  197 

EM:  0.419  0.335  0.464  0.290  0.344  0.427-0.271-0.297-0.186  805 

EN:  0.384  0.315  0.483  0.317  0.397  0.489-0.323-0.366-0.295  781 

FC:  0.428  0.385  0.550  0.344  0.435  0.517-0.280-0.365-0.223  727 

GMG:  0.463  0.362  0.524  0.310  0.388  0.478-0.317-0.346-0.195  393 

0.217  0.182  0.218  0.158  0.248  0.294-0.240-0.277-0.213  554 

IIH-I:  0.248  0.278  0.215  0.195  0.240  0.316-0.385-0.413-0.280  320 

BT/MM:  0.297  0.250  0.407  0.241  0.349  0.318-0.274-0.240-0.214  837 

OS:  0.527  0.460  0.558  0.337  0.423  0.511-0.305-0.363-0.266  622 

RM:  0.461  0.437  0.467  0.320  0.375  0.461-0.219-0.255-0.112  250 

13F:  0.507  0.468  0.582  0.369  0.473  0.556-0.391-0.407-0.333  819 

19K:  0.148  0.139  0.111  0.098  0.141  0.132-0.135-0.127-0.113  1106 

272-1:  0.479  0.409  0.491  0.305  0.411  0.524-0.369-0.372-0.354  484 

732:  0.476  0.493  0.602  0.344  0.412  0.537-0.245-0.271-0.243  421 
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Appendix  G 

Factor  Analysis  Results 


G-0 


Table  G-1 


Estimated  True-Score  Factor  Scores: 


“EIGENVALUE  SUMMARY  REP0RT-- 


FACTOR 

EIGENVALUE 

PCT  OF 

1 

10.18589 

53.61 

2 

1.988704 

10.47 

3 

1.724733 

9.08 

4 

1.159964 

6.11 

5 

.69208 

3.64 

6 

.630049 

3.32 

7 

.449483 

2.37 

8 

.40335 

2.12 

9 

.296031 

1.56 

10 

.266105 

1.4 

11 

.233932 

1.23 

12 

.212813 

1.12 

13 

.206597 

1.09 

14 

.196621 

1.03 

15 

.166402 

.88 

16 

.119208 

.63 

17 

.100537 

.53 

18 

.09325 

.49 
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CUM  PCT 


53.61 

64.08 

73.15 

79.26 

82.9 
86.22 
88.58 
90.71 

92.26 
93.67 

94.9 
96.02 
97.1 
98.14 
99.01 
99.64 
100.17 
100.66 
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Appendix  H 

Mean  Predicted  Performance 
by  Test  Condition  for  Eight  Replications 


H-0 


Table  H-1 


Mean  Predicted  Performance 
by  Test  Condition  for  Eight  Replications 


Conditions 


A1 

A2 

A3 

A4 

A5 

A6 

A7 

A8 

Replications 

1 

.60 

.68 

.64 

.73 

.66 

.68 

.74 

.76 

2 

.56 

.62 

.58 

.65 

.60 

.59 

.66 

.71 

3 

.61 

.63 

.65 

.70 

.60 

.65 

.68 

.76 

4 

.53 

.65 

.62 

.69 

.58 

.59 

.69 

.73 

5 

.64 

.64 

.64 

.67 

.64 

.63 

.71 

.73 

6 

.66 

.71 

.67 

.80 

.65 

.69 

.76 

.78 

7 

.54 

.57 

.53 

.60 

.55 

.58 

.65 

.65 

8 

.66 

.70 

.68 

.73 

.68 

.70 

• 

VO 

.79 
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Table  1 


ASVAB,  CAT-ASVAB,  Alternate  Batteries,  and  Their  Individual  Tests 


Test 


Battery^ 


ASVAB/ CAT-ASVAB 

General  Science 
Arithematic  Reasoning 
Work  Knowledge 
Paragraph  Comprehension 
Numerical  Operations 
Coding  Speed 

Auto  and  Shop  Information 
Mathematical  Knowledge 
Mechanical  Comprehension 
Electronics  Information 

ECAT 

Mental  Counters 
Sequential  Memory 
Figural  Reasoning 
Integrating  Details 
Assembling  Objects 
Spatial  Orientation 
One-Hand  Tracking 
Two-Hand  Tracking 
Target  Identification 


Si 

^2 

23 

S4 

fis 

p 

ca 

p 

ca 

ca 

p 

ca 

p 

ca 

ca 

p 

ca 

p 

ca 

ca 

p 

ca 

p 

ca 

ca 

p 

cn 

p 

cn 

cn 

p 

cn 

p 

cn 

cn 

p 

ca 

p 

ca 

ca 

p 

ca 

p 

ca 

ca 

p 

ca 

p 

ca 

ca 

p 

ca 

p 

ca 

ca 

cn 

cn 

cn 

cn 

p 

cn 

cn 

cn 

cn 

p 

cn 

cn 

p 

cn 

cn 

Mode  of  Administration  for  Each  Teat; 

p  »  paper-and-pencil 
ca  =  computer-adaptive 
cn  «  computer  non-adaptive 

cnp  =  computer  non-adaptive  with  response  pedestal 


Table  2 


Test  Battery  and  Time  Limits  for  Each  Treatment 


Testing  Condition^  Battery  Time  (minutes) 


Ai 

A2 

A3 

A4 

A5 

Ae 

A7 

As 


B2 

B3 

B4 

Bl 

B3 

B4 

B5 


100 

100 

135 

135 

180 

180 

180 

180 


^Legend  for  Teatina  Conditions; 

A,:  P&P-ASVAB,  100  min. 

aI:  cat  ASVAB,  100  min. 

Aj:  P&P-ASVAB  +  P&P-ECAT,  135  min. 

A/:  CAT  ASVAB  +  non-pedestal  SCAT,  135  min. 

AcJ  P&P-ASVAB,  180  min. 

Ii,i  P&P-ASVAB  +  P&P-ECAT,  180  min. 

A7S  CAT-ASVAB  +  non-pedestal  ECAT,  180  min. 
Ag;  CAT  ASVAB  +  full  ECAT,  180  min. 


Table  3 


Battery  Contrasts 


SOW 

Task  No. 

Contrasts^ 

Expected  Completion 

Time  (minutes) 

4.4.1 

Aj  VS  A2 

100 

4.4.2 

A3  VS  A4 

135 

4.4.3 

Ag  vs  Ag 

180 

4.4.4 

Ag  vs  A7 

180 

4.4.5 

A7  vs  Aq 

180 

4.4.6 

Ag  vs  Ag 

180 

^Legend  for  Testing  Conditions s 


A-:  PfiP-ASVAB,  100  min. 

A2:  CAT  ASVAB,  100  min. 

A3:  P&P-ASVAB  +  P&P-ECAT,  135  min. 

A^:  CAT  ASVAB  +  non-pedestal  ECAT,  135  min. 
Aj:  P&P-ASVAB,  180  min. 

A^:  P&P-ASVAB  +  P&P-ECAT,  180  min. 

Kjt  CAT-ASVAB  +  non-pedestal  ECAT,  180  min. 
Agt  CAT  ASVAB  +  full  SCAT,  180  min. 


Coefficients  for  the  Six  Planned  Comparisons 


Table  5 


Experimental 

Design 

Testing 

Condition : 

Replications  (Blocks) : 

®2  • •  •  Bm 

[Bi(lOO)] 

PPi^2  •••  PBi,m 

A2  [B2(100)] 

^^2,2  •••  BP2^M 

A3  [B3(135)] 

^^3,1 

PP3,2  •••  ^B3,M 

A4  [B4(135)] 

^^4,2  ••• 

A5  [Bi(180)] 

PP5,1 

^^5,2  •••  PB5,M 

Ag  [B3(180)] 

^^6,2  •••  PB6,M 

A7  [B4(180)] 

^^7,2  •••  PP7,M 

Ag  [B5(180)] 

PB8,2  •••  PB8,M 

Ag  [BgClSO)] 


Table  6 


Joint-Service  Schools 


Abbreviation  Title 


Naw: 

AC 

AE 

AMS 

AO 

AV 


EN 
EM 
FC 
GMG 
BT/MM 
OS 
RM 

Air  Force! 

272  Air  Traffic  Controller 

732  Personnel  Specialist 

Army.?. 

13F  Ft.  Sill  -  Artillery  Specialist 

IIH(H)*  Ft.  Banning  -  Tow  Missile  Specialist 

IIH(I)* 

19K  Ft.  Knox  -  Ml  Tank  Crewman 


*  IIH  data  was  separated  into  two  samples  reflecting  separate  school  tracks,  I 
and  H.  Each  track  sample  was  analyzed  as  if  it  were  a  separate  school  sample. 


Air  Controlman 

Aviation  Electrician's  Mate 

Aviation  Structural  Mechanic-Structures 

Aviation  Ordinanceman 

Avionics  Total,  consisting  of; 

Aviation  Electronics  Technician  (AT) 

Aviation  Fire  Control  Technician  (AQ) 

Aviation  Antisubmarine  Warfare  Technician  (AX) 
Engineman 
Electrician's  Mate 
Fire  Control  Technician 
Gunner's  Mate 

Boiler  Technician/Machinist's  Mate 

Operations  Specialist 

Radioman 


Table  7 


Optimal  Test  Times  (minutes)  for  the  Eight 
Testing  Conditions 


Testing 

Condition: 

Test :  : 

A2  s 

A3: 

A^  * 

^5* 

Ae: 

An  e 

A3: 

Bl 

®2 

B3 

B4 

Bl 

B3 

B4 

Bs 

GS 

9.06 

7.84 

11.11 

8.884 

15.574 

14.308 

9.990 

9.972 

AR 

16.22 

14.04 

17.13 

14.595 

26.172 

22.046 

16.908 

13.497 

WK 

10.45 

8.88 

11.59 

10.017 

16.453 

14.596 

12.854 

11.783 

PC 

4.58 

17.95 

8.04 

22.512 

21.500 

16.499 

32.188 

28.761 

NO 

3.35 

3.11 

3.77 

3.426 

4.924 

4.552 

4.701 

4.137 

cs 

7.13 

5.42 

7.84 

5.854 

9.888 

9.198 

7.969 

8.113 

AS 

8.28 

6.38 

9.38 

7.365 

12.744 

11.604 

8.840 

8.108 

HK 

14.97 

7.78 

15.94 

8.103 

20.314 

18.445 

9.126 

8.653 

MC 

12.04 

14.93 

14.00 

17.318 

25.834 

21.457 

23.070 

19.648 

El 

13.92 

13.67 

17.13 

15.536 

26.597 

23.584 

20.381 

18.641 

CT 

3.612 

5.187 

5.146 

SM 

4.753 

7.311 

6.681 

FR 

8.13 

5.771 

9.951 

9.093 

8.958 

ID 

.475 

2.165 

1.588 

AO 

3.33 

2.793 

4.770 

4.824 

2.885 

SO 

7.59 

3.986 

8.991 

5.423 

4.564 

T1 

8.385 

T2 

5.938 

TI 

4.541 

Nat®.  The  batteries  Bg,  and  Bj  are  computer  administered.  At  the  completion 
Of  one  test/  the  examinee  may  proceed  immediately  to  the  next  without  reoard  to 
time  limits*  Therefore,  the  time  limits  for  the  individual  tests  do  not 
constrain  total  administration  time,  and  expected  administration  time  is  the 
appropriate  measure. 


Table  a. 


CAT-ASVAB  Original  Test  Lengths,  and  Lengths  and  Reliabilities  for 
Proportionally  Lengthened  tests  to  achieve  100  minutes 

Testing  Time 


Orifiinal 

—  —  — —  —  —  —  —  — —  j;  —  —  —  — ^  ^ 

Revised 

Test ; 

Length : 

Reliability: 

Length : 

Reliability 

GS: 

4.712 

.88 

5.591 

.90 

AR: 

20.973 

.92 

24.890 

.94 

WK: 

4.112 

.91 

4.880 

.92 

PC: 

13.604 

.84 

16.144 

.87 

NO; 

3.000 

.86 

3.560 

.88 

CS: 

7.000 

.82 

8.307 

.84 

AS: 

6.196 

.92 

7.353 

.93 

MK: 

9.461 

.93 

11.228 

.95 

MC: 

10.787 

.87 

12.801 

.88 

El : 

4.419 

.82 

5.244 

.84 

SUMS: 

84.264 

99.998 

Table  9 


Matrix  of  Estimated  Predictor  True-Score  Intercorrelations 


Z-Score  Means,  Standard  Deviations,  and  Intercorrelations 
among  ASVAB  and  SCAT  Tests,  Corrected  for  Restriction 
in  Range  and  Attenuation 


GS: 

AR: 

WK: 

PC: 

NO: 

CS: 

AS: 

MK: 

MC: 

El: 


AR:  WK:  PC: 

0.723  0.838  0.810 
0.689  0.761 
0.953 


NO:  CS:  AS: 

0.324  0.300  0.623 
0.550  0.473  0.477 
0.373  0.386  0.511 
0.522  0.520  0.455 
0.762  0.056 
0.071 


MK: 

MC: 

El: 

0.648 

0.798 

0.820 

0.822 

0.763 

0.636 

0.568 

0.669 

0.686 

0.655 

0.680 

0.654 

0.574 

0.282 

0.188 

0.483 

0.280 

0.195 

0.231 

0.778 

0.884 

0.607 

0.477 

0.871 

CT: 


SM: 


FR: 


ID: 


AO: 


SO: 


Tl: 


T2: 


TI: 


MEAN: 


GS:  0.437  0.442  0.612  0.605 

AR:  0.655  0.638  0.725  0.681 

WK:  0.395  0.436  0.568  0.506 

PC:  0.463  0.503  0.606  0.523 

NO:  0.433  0.416  0.373  0.305 

CS:  0.417  0.421  0.358  0.310 

AS:  0.246  0.205  0.381  0.459 

MK:  0.600  0.581  0.656  0.609 

MC:  0.531  0.485  0.685  0.727 

El:  0.351  0.318  0.521  0.569 

CT:  0.748  0.681  0.661 

SM:  0.674  0.606 

FR:  0.739 

ID: 

AO: 

SO: 

Tl: 

T2: 

TI: 


0.565  0.603-0.343-0.394-0.366 
0.608  0.660-0.351-0.385-0.304 
0.465  0.529-0.287-0.339-0.287 
0.478  0.542-0.307-0.340-0.291 
0.279  0.275-0.246-0.222-0.211 
0.320  0.299-0.245-0.250-0.232 
0.468  0.487-0.313-0.372-0.266 
0.546  0.585-0.302-0.312-0.258 
0.700  0.726-0.465-0.526-0.399 
0.562  0.585-0.350-0.412-0.296 
0.680  0.620-0.451-0.441-0.345 
0.596  0.573-0.393-0.389-0.333 
0.710  0.692-0.428-0.437-0.351 
0.787  0.720-0.460-0.471-0.389 
0.722-0.464-0.494-0.431 
-0.455-0.479-0.339 
0.879  0.439 
0.437 


0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 

0.000 


SD: 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 

1.000 


Table  10 


Actual  Enrollment  Numbers  and  Assignment  Quotas  by  School 


School 

Actual 

Enrollment 

School 

Assignment  Quota 

AC 

607 

1 

AE 

1016 

2 

AMS 

1067 

2 

AO 

1158 

3 

AV 

2138 

5 

EM 

4586 

10 

EN 

1334 

3 

FC 

874 

2 

GMG 

346 

1 

IIH(H) 

872 

2 

IIH(I) 

331 

1 

MM 

6422 

15 

OS 

1176 

3 

RM 

1410 

3 

13F 

1581 

4 

19K 

2307 

5 

272 

470 

1 

732 

868 

2 

SUBTOTAL 

28563 

65 

REJECTS^ 

35 

TOTAL 

28563 

100 

^  To  accurately  reflect  the  population  selection  ratio  of  *652  (based  on  FY  92 
data),  the  assignment  algorithm  optimally  classified  65  of  the  100  subjects  to 
schools  while  allocating  the  remaining  35  subjects  to  the  reject  category. 


Table  11 


Job  Values  by  Job  Complexity  Ratings  for  the 
17  Occupational  Specialties 


Job  Complexity 

Occupational 

Per  Person 

Mean  Job 

Level 

Specialty 

Job  Value 

Value  by 
Complexity 

1 

27230 

128,698.51 

AC 

130,426.90 

OS 

127,791.98 

128,972.46 

2 

AMS 

126,997.49 

GMG 

121,187.10 

73230 

130,418.76 

AE 

130,176.66 

AV 

132,757.99 

MM 

140,131.90 

AO 

125,129.15 

• 

EN 

133,955.73 

EM 

135,456.19 

FC 

135,910.52 

RM 

128,018.92 

130,921.86 

3 

13F 

104,406.45 

IIH 

106,674.27 

19K 

103,639.08 

104,906.60 

Table  12 


Military  and  Civilian  Pay  by  Job  Complexity  Category 


MILITARY 

CIVILIAN 

Complexity 

Category 

Job 

Mean 

Monthly 

Salary 

Mean 

Salary  by 
Category 

Job 

Mean^ 

Monthly 

Salary 

Mean 
Salary  by 
Category 

1 

27230 

1,928 

6,829 

OS 

1,915 

Attorney 

8,056 

AC 

1.954 

1,932 

Physician^ 

9,287 

2 

AMS 

1,930 

GMG 

1,816 

73230 

1,954 

AE 

1,950 

AV 

1,989 

MM 

2,100 

AO 

1,875 

EN 

2,007 

Auto  Mech 

3,100 

EM 

2,029 

Repairman 

2,865 

2,982 

RM 

1,918 

1,962 

FC 

2,036 

Oerks 

1475 

3 

13F 

1,564 

Mail  Hndlr 

1437 

IIH 

1,598 

Tel  Operatr 

1,652 

1439 

19K 

1453 

1472 

File  aerk 

1498 

Note,  The  civilian  jobs  appearing  in  this  teble  represent  our  attempt  to  match  as  many  civilian  jobs  as  appeared  in  the  job  complexity 
categories  of  Hunter,  Schmidt,  &  Judiesch  (1990,  Table  5,  p.  34). 

^  Mean  monthly  salaries  obtained  from  Employers*  Group  Salary  Survey  covering  Southern  California  (November  1993). 

^  Mean  monthly  salary  computed  from  data  presented  in  Hunter,  Schmidt,  &  Judiesch  (1990)  based  on  Ovens  (1987)  national  survey  of  the 
earnings  of  nonsalaried  Physicians. 


Table  13 


Job  Values  by  Job  Complexity  Ratings  for  the  16  MOSs  from 

Bobko  &  Donnelly  (1988) 


Job  Complexity  Occupational  Per  Person  Mean  Job 

Level  Specialty  Job  Value  Value  by 

Complexity 


2  71L 

55B 
5  IB 
94B 
63B 
9  IB 
95B 
64C 
27E 
76Y 
67N 


119,062.54 

101,062.50 

103,324.33 

106,697.44 

115,087.37 

133,339.38 

108,206.83 

290,984.77 

102,984.80 

125,089.95 

114,310.94  129,104.62 


13B  100,524.31 
12B  103,081.87 
IIB  108,855.34 
19E  128,764.09 


110,306.40 


Table  14 


Complexity  and  SDy  Values  for  the 
17  Occupational  Specialties 


Job 

Complexity 

Level 

Occupational 

Specialty 

SDp 

Value 

Per  Person 
Job  Value 

SDy 

Value 

1 

27230 

.48 

128,698.51 

61,131.79 

AC 

.48 

130,426.90 

61,952.78 

OS 

.48 

127,791.98 

60,701.19 

2 

AMS 

.32 

126,997.49 

40,385.20 

GMG 

.32 

121,187.10 

38,537.50 

73230 

.32 

130,418.76 

41,473.17 

AE 

.32 

130,176.66 

41,396.18 

AV 

.32 

132,757.99 

42,217.04 

MM 

.32 

140,131.90 

44,561.94 

AO 

.32 

125,129.15 

39,791.07 

EN 

.32 

133,955.73 

42,597.92 

EM 

.32 

135,456.19 

43,075.07 

FC 

.32 

135,910.52 

43,219.55 

RM 

.32 

128,018.92 

40,710.02 

3 

13F 

.19 

104,406.45 

20,150.44 

IIH 

.19 

106,674.27 

20,588.13 

19K 

.19 

103,639.08 

20,002.34 

Tabic  15 


An  Overview  of  Steps  to  Compute  Total  Net  Benefits  (TNB) 


Complexity 

Category 

Job 

SDy 

WMPP^  by  Battery 

Nper  School 

2 

AUjotaKPer  School) 
bv  Battery 

(a) 

(b) 

(a) 

(b) 

1 

44E 

49204.42 

1.255 

1.207 

148 

9,139,229 

8,789,681 

AC 

61952.78 

1.255 

1.207 

607 

47,194,699 

45,389,642 

45510 

56141.31 

1.255 

1.207 

117 

8,243,509 

7,928,220 

2 

2800 

25383.92 

595 

.668 

646 

9,756,817 

10,953,872 

29N 

31502.84 

595 

.668 

193 

3,617,629 

4,061,472 

SK 

34181.79 

595 

.668 

636 

16,340,853 

18,345,697 

45214 

39986.41 

595 

.668 

1023 

24,339,128 

27325,273 

3 

EO 

22991.07 

.357 

.348 

349 

2,864,526 

2792,311 

5831 

22814J1 

.357 

.348 

118 

961,084 

936,855 

91F 

19769.70 

.357 

.348 

186 

1,312,748 

1279,653 

^UjotaKa)^" 

4,923,020,576 

^UxotaKb)^  “ 

5,378,991,338 

TNB  -  AUjotaKb) 

-AUiotaKa)  “ 

455.970,762 

Note.  Battery  a  =  Paper  &  Pencil,  Battery  b  =  CAT. 

^  WMPP  values  arc  the  same  for  each  school  in  a  complexity  category  but  different  for  each  battery. 
^  This  is  the  product  of  ADg  •  Np^^  School  battery. 

^  These  arc  column  sums. 


Ovciview  of  Procedures  for  Present  Value  Analyses 
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Initial  Setup  Costs  for  Various  Batteries  (in  $K) 


.^tc.  Data  appeanng  in  Victno,  Hetter,  Moreno,  Rafacz,  Segnll,  and  Unpingpo  (1993)  served  as  a  basis  for  this  table.  Interpolations  or  extrapolations  based  on  their  cost  data  (based  on  a 
2-hour  test)  and  our  test  completion  times  provided  the  data  for  this  table. 

^  These  costs  vary  directly  with  test  completion  time.  All  other  costs  arc  as  presented  in  Vicino  ct  al. 

2 

Vkino  ct  al.  present  their  cost  estimates  in  1993  dollars.  The  adjusted  totals  in  this  row  represent  an  additional  3%  per  year  to  place  them  in  1995  dollars. 


Table  18 


Mean  Classification  Efficiency  Indices’  Across  Replications 
for  Each  Testing  Condition  (TC) 


TC 

Battery 

Time 

Phi 

R' 

r' 

c 

AA 

6 

100 

.9926 

.6908 

.9266 

.1872 

.3948 

.3912 

A2 

B2 

100 

1.3079 

.7178 

.8919 

.2360 

.4977 

.4065 

^3 

83 

135 

1.1039 

.7029 

.9101 

.2108 

.4446 

.3981 

^4 

84 

135 

1.4738 

.7302 

.8728 

.2604 

.5492 

.4135 

Bl 

180 

1.1366 

.7045 

.9102 

.2111 

.4452 

.3990 

^6 

83 

180 

1.1843 

.7098 

.9011 

.2232 

.4707 

.4020 

180 

1.6000 

.7398 

.8598 

.2762 

.5825 

.4178 

As 

85 

180 

1.7789 

.7449 

.8399 

.2980 

.6285 

.4218 

’  notation  for  Indices  Above; 

Phi:  Horst's  index  of  differential  validity. 

R':  The  average  multiple  correlation  across  schools. 

r' :  The  average  correlation  between  regression  equations  across  schools. 

C:  The  multiplier  for  Brogden's  Allocation  Average  Index. 

AA:  Brogden's  Allocation  Average. 

G:  The  Allocation  Average  using  a  single  factor  as  a  predictor. 


Table  19 


Analysis  of  Variance  on  Mean  Predicted  Performance  Values 


sv 

SS 

df 

MS 

F 

A 

.1356 

7 

.0194 

50.46*** 

B 

.1179 

7 

.0169 

A  X  B 

.0188 

49 

.0004 

Total 

.2723 

63 

***£<.001. 


Table  20 


Significance  Tests  for  Planned  Comparisons 


SOW 

SV 

d 

t 

4.4.1 

A1  V.  A2 

.0486 

4.96*** 

4.4.2 

A3  V.  A4 

.0697 

7.11*** 

4.4.3 

A5  V.  A6 

.0188 

1.92* 

4.4.4 

A6  V.  A7 

.0714 

7.29*** 

4.4.5 

A7  V.  A8 

.0314 

3.20** 

4.4.6 

A6  V.  A8 

.1028 

10.49*** 

.0098 

*E<.05/  one-tailed. 
**E<.01,  one-tailed. 
***^<.001,  one-tailed. 


\vcnige  Mean  Predicted  Performance  (MPP) 
by  Testing  Condition  and  Research  Question 


in  parentheses  are  the  average  MPPs  multiplied  by  the  Brogden  factors  appearing  in  Table  23, 


Table  23 


Obseirved  and  Esbi.ma'ted  Mean  Predicted  Performance  Values  ( Broaden 's  AA) 


Broaden  Multiplication  Observed 


Factor 


N  Jobs: 

18 

622 

18 

622 

Treatment 
Condition : 

.40 

.62 

1.550 

. 

a\ 

o 

.930 

Aj 

.50 

.78 

1.560 

.65 

1.014 

^3 

.45 

.69 

1.533 

.63 

.966 

.55 

.86 

1.564 

.70 

1.095 

.45 

.70 

1.556 

.62 

.964 

^6 

.47 

.74 

1.574 

.64 

1.008 

^7 

.58 

.91 

1.569 

.71 

1.114 

.63 

.98 

1.556 

.74 

1.151 

Utilities  (in  $M) 

by  Testing  Condition  and  Research  Question 


Present  Value  of  Total  Net  Benefits  (or  Coste) 

Adjusted  for  the  Brogden  Factor  by  Fiscal  Year  and  Research  Question 
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^  Values  in  this  row  arc  the  initial  setup  cost  (i.c.,  the  investment  cost)  of  the  alternative  hypothesized  as  more  effective.  Since  we  did  not  discount  these,  the  present  value  analyses  ircai  mem 
as  thought  they  all  occur  at  the  beginning  of  199S. 

♦  Hie  net  piesent  values  in  this  row  aie  column  sums.  These  include:  roughly  towr  yea**  per  cohort  (U.,  the  3X6  years  that  the  typical  enlistee  remains  in  the  military),  for  four  cohorts  (i.e., 
the  typical  number  of  years  an  ASVAB  form  remains  in  service),  and  the  initial  setup  cost. 


